Funnels, metrics and retention

0:00 Funnels, metrics and retention

Sign-ups are up forty percent this month. Great news? Maybe. Or maybe you're pouring water into a leaking bucket, and the leak is getting bigger. In this lecture, you'll learn to measure the whole customer journey, find your aha moment, read a cohort retention table, pick a North Star metric, and recognise real product-market fit. And because AI products have a special trap, novelty that fades, you'll learn the extra metrics that show whether the AI is actually doing the job.

0:35 Why it matters

Why does this matter? Because growth without retention is an illusion. If customers leave as fast as they arrive, you're spending money to stand still. And averages hide the truth. Total revenue can rise while every new group of customers is less loyal than the last. Here's the key idea. Retention is the foundation. Acquisition only pays off if customers stay, and the most honest measure of value is whether people keep coming back without being pushed.

1:08 AARRR: pirate metrics

Let's use a framework. Dave McClure's AARRR, often called pirate metrics. Acquisition: how people find you. Activation: their first real experience of value. Retention: whether they come back. Referral: whether they tell others. And revenue: whether they pay. Think of it like a restaurant again. Acquisition is people walking in. Activation is the first bite that makes them smile. Retention is coming back next week. Referral is bringing friends. Revenue is the bill. If the first bite is disappointing, no amount of marketing outside will fix it.

1:46 Aha moment and North Star

Now the aha moment. That's the action that most predicts whether someone will stay. For a note-taking app, it might be creating three notes in the first week. For a marketplace, completing a first successful transaction. You find it by comparing users who stayed with users who left, and asking what the stayers did early that the leavers didn't. Then you redesign onboarding to get every new user there faster. Next, pick a North Star metric: one number that captures the value customers receive, like weekly successful bookings, not vanity numbers like page views.

2:27 Worked example 1: a learning app (illustrative)

A simple worked example, illustrative. A language-learning app in Karachi sees ten thousand downloads a month, but only a few hundred users still active after a month. The funnel shows the drop happens at activation. Most users never finish a first lesson. The team compares stayers and leavers. Stayers finished a five-minute lesson on day one and got a streak reminder the next morning. So they shorten the first lesson, add a friendly reminder, and remove the account creation step until after the first lesson. Activation climbs, and so does month-one retention.

3:07 Reading a cohort table

Now the most important tool: the cohort retention table. Each row is a group of customers who started in the same month. Each column shows what share are still active after one, two, three months and so on. Two things to look for. First, do the curves flatten? A healthy product loses some people early, then settles at a stable core. Second, compare rows. If newer cohorts retain worse than older ones, something's deteriorating: your targeting, your onboarding or your quality, even if total revenue is growing. This table tells the truth that averages hide.

3:48 AI product metrics

Here's the AI trap. AI products often see a burst of use in week one, when everything feels magical, followed by a slump once the novelty wears off. I call it novelty retention. So judge retention on cohorts after eight to twelve weeks, not on launch excitement. And add AI-specific metrics. Task success rate: the share of outputs users accept without major edits. Edit rate or rework time. Escalation rate: how often a human has to take over. Cost per successful task, which links usage to unit economics. And weekly active use of the core action, not just logins.

4:31 Worked example 2: AI meeting notes (illustrative)

Now a realistic scenario, illustrative. An AI meeting-notes startup in London sees sign-ups grow every month. The founder, Ben, is delighted, until he builds the cohort table. Newer cohorts are retaining worse. Why? He segments task success by meeting type. It's high for English-only meetings but much lower for mixed-language meetings, which are common among his fast-growing customers in the Gulf. So he pauses paid acquisition in that segment, improves language handling, and adds a fix this summary button to measure edit rates. He resumes growth only when task success and month-three retention recover for new cohorts.

5:13 Watch me: a cohort table

Watch me build a cohort table from an export. I download a list of customers with their start date and the dates they were last active. In a spreadsheet, I add a column for the start month, and then columns for months one to six, using a formula that checks whether each customer was active in that month. A pivot table gives me the share of each cohort still active. I apply colour shading from green to red. Then I read it: do the curves flatten, and are newer cohorts better or worse? Finally, I add the Sean Ellis question to a short survey for active users: how would you feel if you could no longer use this product?

6:05 Product-market fit signals

About that survey. The Sean Ellis question asks active users how they'd feel if they could no longer use your product: very disappointed, somewhat disappointed, not disappointed, or no longer using it. Many teams treat around forty percent very disappointed as an encouraging signal of product-market fit. Treat that as a heuristic, not a law. The richer insight is who says very disappointed. Study those people, find what they have in common, and build to get more users like them. Also, talk to churned customers. A short, respectful call often reveals the fix faster than any dashboard.

6:47 Common mistakes

Let's go through the common mistakes. Celebrating sign-ups while ignoring retention. Tracking too many metrics, so none drives decisions. Using vanity metrics as the North Star. Looking only at averages, not cohorts. For AI products, judging on launch-week excitement and measuring logins instead of successful tasks. And never speaking to the people who left. A simple weekly dashboard with five to seven numbers, reviewed at the same time every week, beats an elaborate one nobody opens.

7:20 Recap and try this now

Let's recap. Measure the whole journey with AARRR, find your aha moment, and onboard people there fast. Choose a North Star metric that reflects real value. Read cohort tables: look for flattening curves and compare newer cohorts with older ones. For AI products, watch for novelty retention and measure task success, edits, escalations and cost per successful task. Your try this now: build a cohort table from your data, or design one for your idea, and write down your aha moment hypothesis and North Star metric. Next module: building a simple financial model.