A creative testing framework

Video lecture

A creative testing framework: turning ads into a learning system

11 chapters · about 8 min · full transcript

Coming soon

Chapter 1 of 11

Creative testing framework

Hypotheses
One variable at a time
Testing methods
Reading results honestly
A quarterly testing matrix

The narrated lecture is in production

Every chapter is scripted and ready. Browse the chapters and read the full transcript now — the video will appear here when it’s published.

Chapters

Transcript of the narration, chapter by chapter.

0:00 Creative testing framework

Ask most teams how they test creative and you'll hear something like: we launched lots of ads and one did really well. Great. Why did it win? Silence. And without the why, you can't do it again. In this lecture you'll learn how to turn creative into a learning system. You'll write proper hypotheses, isolate one variable at a time, choose the right testing method, decide your metric and budget before launch, read results without fooling yourself, including a quick significance check, and plan a whole quarter of testing with a simple matrix.

0:40 Step 1: hypothesis

Step one is a hypothesis. It names the variable, the expected effect and the reason. For example: a problem-first hook, tired of chasing invoices, will produce a higher hook rate and a lower cost per sign-up than a product-first hook, because our audience is more motivated by pain than by features. Notice the because. It's the part that teaches you something, win or lose. If the problem-first hook loses, you've learned that your audience responds differently than you thought. That's valuable. A test without a hypothesis is just spending money and hoping.

1:20 Step 2: one variable

Step two: isolate one variable. There's a natural order, from big ideas to small tweaks. Concept: testimonial versus demo versus founder story. Angle: saving time versus saving money. Hook: three different openings with the same body. Format: video versus static versus carousel. Creator: two creators with the same script. And offer: free delivery versus ten percent off. If you change the concept, the hook and the offer all at once, you might find a winner, but you won't know why. And big ideas first, always. A new concept can halve your cost. A new button colour almost never will.

2:03 Step 3: method

Step three: choose a testing method. Platform A/B test tools, like Meta's Experiments and TikTok's split test, divide audiences so each person sees only one version. That's the cleanest comparison. Putting several ads in one ad set is faster and cheaper, but delivery favours some ads early, so results are directional, not controlled. And dynamic or flexible creative, where the platform mixes headlines, texts and assets, is great for optimising, but it blurs the reasons behind results. So use clean A/B tests for important questions, and multi-ad ad sets for quick exploration.

2:43 Steps 4–5

Step four: decide your metric and budget before launch. Pick one primary metric that reflects the goal: cost per purchase or ROAS for sales, cost per qualified lead for lead generation. Hook rate, hold rate and click-through are diagnostics, not the verdict. Set a minimum spend per variant, a common rule of thumb is at least a couple of times your target cost per acquisition, and prefer a meaningful number of conversions. And run for around five to seven days to smooth out weekday effects. Then step five: read carefully. Small differences on small samples are usually noise.

3:26 Example 1: Dubai course creator (illustrative)

Let's do a simple worked example with illustrative numbers. An online course creator in Dubai tests two hooks for seven days at fifteen hundred dirhams each. Hook A: I failed my first launch. Hook B: five steps to launch a course. Hook A gets thirty-eight sign-ups, about thirty-nine dirhams each. Hook B gets twenty-one, about seventy-one dirhams each. The platform's A/B tool reports a high likelihood that A wins. So she scales A and briefs three new story-led hooks for the next round. Notice the learning: personal failure stories beat instructional openings for her audience. That's reusable knowledge.

4:09 Is it real?

How do you know a difference is real, not luck? The platform A/B tools give you a confidence figure, so use it. For landing page or form tests, where you know clicks and conversions for each version, there's a quick check you can run in Google Sheets. It's called a two-proportion z test, and the five formulas are in the lesson text. You get a p-value. Below zero point zero five is a common threshold for unlikely to be chance. For example, two thousand clicks each with sixty versus forty conversions is borderline. Two hundred clicks each with six versus four is pure noise. Treat it as a sense check, not gospel.

4:58 Quarterly testing matrix

Now let's plan a quarter with a testing matrix. Round one asks which concept wins, testimonial, demo, founder story or offer static, with audience, offer and budget held constant. Round two takes the winning concept and tests three hooks. Round three tests which creator delivers the winning script best. And round four tests the offer, judged on contribution per order, not just conversion rate. Each round's winner becomes the control for the next. It's like a tournament, where every match teaches you something and the champion gets stronger each round. The full matrix is in the lesson text.

5:41 Example 2: Riyadh fashion (illustrative)

Here's a realistic scenario. A Riyadh modest-fashion label runs that matrix on Snapchat and TikTok. In round one, a creator try-on beats the studio lookbook and the offer static. In round two, the hook, what I wear to work in Riyadh summer, beats new collection is here. In round three, two of three creators beat the brand's own account, so they renew those creators' usage rights. And in round four, free delivery beats ten percent off on contribution per order, even though the discount converted slightly better. The written learning: situational, work-life hooks from real creators win, and delivery offers protect margin.

6:25 Step 6: document and iterate

Step six is the one teams skip: document and iterate. Keep a simple testing log with the date, hypothesis, variants, result, learning and next test. Over months, that log becomes your most valuable asset, a playbook of what your audience responds to. A practical weekly rhythm: Monday, review last week's tests and record learnings. Tuesday, brief new creative based on them. Wednesday and Thursday, launch new tests. And ongoing, move clear winners into scaling campaigns and pause losers at your stop-loss. Common mistakes: testing many variables at once, calling winners after a day, judging sales tests on click-through, and never writing anything down.

7:10 Recap and try this now

Let's recap. Start every test with a hypothesis that includes a because. Test one variable at a time, big ideas first. Use A/B tools for important questions. Decide the primary metric, minimum spend and duration before launch, and be honest about small samples. Plan a quarter as a matrix where each winner becomes the next control, and write every learning down. How do you know it's working? Every live test has a hypothesis, metric and stop date, and your cost per result trends down as spend grows. Here's your try this now. Write three hypotheses for your next tests, each isolating one variable, with the metric, minimum spend and duration.

Why structured testing

Without a structure, creative testing becomes "we launched lots of ads and one did well". You cannot tell why, and you cannot repeat it. A testing framework turns creative into a learning system.

Step 1: Write a hypothesis

A good hypothesis names the variable, the expected effect and the reason:

"A problem-first hook ('Tired of chasing invoices?') will produce a higher hook rate and lower CPA than a product-first hook, because our audience is more motivated by pain than by features."

Step 2: Isolate one variable

Test one major variable at a time:

Level	Variable examples
Concept	Testimonial vs demo vs founder story
Angle	Time-saving vs money-saving
Hook	Three different openings with the same body
Format	Video vs static vs carousel
Creator	Creator A vs Creator B with the same script
Offer	Free delivery vs 10% off

Changing concept, hook and offer at once may find a winner but will not tell you what worked.

Step 3: Choose a testing method

Platform A/B test tools: Meta's A/B test feature and TikTok's split testing divide audiences so each person sees only one version, giving a cleaner comparison.
Multiple ads in one ad set: faster and cheaper, but delivery will favour some ads early, so results are directional, not controlled.
Dynamic or flexible creative: the platform mixes headlines, texts and assets (for example Meta's flexible ad format and Advantage+ creative, or TikTok's automated creative options in Smart+). Useful for optimisation, less useful for learning exactly why something worked.

Step 4: Decide your success metric and budget in advance

Pick one primary metric that reflects the goal: CPA or ROAS for sales, cost per qualified lead for lead generation. Use hook rate, hold rate and CTR as diagnostic metrics.
Set a minimum spend or number of conversions per variant before judging. A frequently used rule of thumb is to let each variant spend at least a couple of times your target CPA before deciding, and to prefer waiting for a meaningful number of conversions.
Set a time frame (for example 5–7 days) to smooth out day-of-week effects.

Step 5: Read results carefully

Statistical caution: small differences on small samples are often noise. Platform A/B tools report a confidence level or likelihood; use it.
Look at the full funnel: a hook with a great hook rate but poor CPA may attract curious viewers who are not buyers.
Segment sensibly: check performance by placement and age only if you have enough data.

Step 6: Document and iterate

Keep a simple testing log:

Date	Hypothesis	Variants	Result	Learning	Next test
1–7 Oct	Problem hook beats product hook	H1 vs H2	H1 lower CPA with a meaningful difference	Pain-led hooks work for this audience	Test three new pain-led hooks

Over months, the log becomes your most valuable asset: a playbook of what your audience responds to.

A practical weekly cadence

Monday: review last week's tests, record learnings.
Tuesday: brief new creatives (in-house or creators) based on learnings.
Wednesday–Thursday: launch new tests.
Ongoing: move clear winners into scaling campaigns; pause clear losers once they hit your stop-loss threshold.

Worked example (illustrative numbers)

An online course creator in Dubai tests two hooks for 7 days at AED 1,500 each:

Hook A ("I failed my first launch…"): 38 sign-ups, CPA ≈ AED 39.
Hook B ("Five steps to launch a course"): 21 sign-ups, CPA ≈ AED 71.

The platform's A/B tool reports a high likelihood that A wins. She scales A and briefs three new story-led hooks to test next.

Common mistakes

Testing many variables at once.
Declaring winners after a day or a handful of conversions.
Judging only on CTR when the goal is sales.
Never writing down learnings.

Hands-on: a creative testing matrix

Plan a quarter of testing as a matrix so each round answers one question and feeds the next:

Round	Question	Variants (one variable)	Held constant	Primary metric	Budget / duration
1	Which concept wins?	Testimonial vs demo vs founder story vs offer static	Audience, offer, budget	Cost per purchase	4 × 2× target CPA, 7 days
2	Which hook for the winning concept?	3 openings, same body	Concept, body, audience	Hook rate → cost per purchase	7 days
3	Which creator delivers the winning script best?	Creator A vs B vs C	Script, hook, offer	Cost per purchase	7–10 days
4	Which offer?	Free delivery vs 10% off	Winning ad	Contribution per order	7 days

Each round's winner becomes the control for the next. Record every round in your testing log.

Hands-on: is the difference real? A quick significance check in Sheets

For conversion-rate comparisons (for example landing-page or lead-form tests where you know clicks and conversions per variant), put clicks and conversions for A in B2:C2 and for B in B3:C3:

Rate A        D2: =C2/B2
Rate B        D3: =C3/B3
Pooled rate   D5: =(C2+C3)/(B2+B3)
z-score       D6: =(D2-D3)/SQRT(D5*(1-D5)*(1/B2+1/B3))
p-value       D7: =2*(1-NORM.S.DIST(ABS(D6),TRUE))

A p-value below 0.05 is a common threshold for "unlikely to be chance". With 2,000 clicks each and 60 vs 40 conversions, p is roughly 0.04 – borderline. With 200 clicks each and 6 vs 4 conversions, p is far above 0.05 – noise. Treat this as a sense check: platform A/B tools, repeated tests and consistency across weeks matter more than one p-value.

Worked example 2: a KSA fashion brand's quarter of testing

A Riyadh modest-fashion label runs the matrix above on Snapchat and TikTok:

Round 1 (concepts): creator try-on beats studio lookbook and offer static on cost per purchase.
Round 2 (hooks): "What I wear to work in Riyadh summer" wins on hook rate and cost per purchase over "New collection is here".
Round 3 (creators): two of three creators beat the brand's own account; the team renews their usage rights.
Round 4 (offer): free delivery beats 10% off on contribution per order, even though 10% off had a slightly higher conversion rate.

Learning recorded: "Situational, work-life hooks from real creators outperform product announcements; protect margin with delivery offers."

How to measure success

Every live test has a written hypothesis, primary metric and stop date.
The testing log grows every week and next briefs cite it.
Over a quarter, the account's cost per result trends down or holds while spend grows.

Start with a hypothesis that names the variable, expected effect and reason.
Test one major variable at a time, from concept and angle down to hooks and offers.
Set the primary metric, minimum spend and time frame before launching; be cautious with small samples.
Log every test and use learnings to brief the next round of creative.

Write three hypotheses for your next creative tests, each isolating one variable. Define the primary metric, minimum spend per variant and test duration for each.

Enrol for free to save your progress

Reading is always free. Enrol to keep your place, take the final assessment and earn a verifiable certificate.