Tokens, context windows, training and inference

0:00 Four terms that explain AI behavior

Why does an AI assistant forget your brand rules halfway through a long chat? Why does it describe a platform policy that changed months ago? And why does it get a character count wrong for an ad headline, even though it writes beautifully? Four terms explain almost every one of those why did it do that moments: tokens, the context window, training and inference. You don't need to be an engineer to understand them. In this lecture you'll get a simple mental model for each, see two examples of them causing real problems, watch me rescue a drifting chat, and leave with three habits that fix most of these issues.

0:48 Why it matters

Why should a marketer care about these words? Because they explain the behavior you see every day. When you understand tokens, you'll know why character limits need checking and why some languages hit limits sooner. When you understand the context window, you'll know why long chats drift and how to fix it in thirty seconds. When you understand training, you'll know why the model doesn't know last month's policy change. And when you understand inference, you'll know why the same question can get different answers. Fewer surprises. Better results on long projects. And smarter use of your plan's limits.

1:31 Tokens

First, tokens. Models don't read words the way we do. They break text into tokens: small chunks that are often a short word, part of a longer word, or a punctuation mark. Marketing might be one token. An unusual brand name might be three. Why does that matter? Two reasons. Limits and pricing for many AI tools are measured in tokens, not words, and text in scripts like Urdu or Arabic often uses more tokens for the same meaning, so long prompts in those languages can hit limits sooner. And because models see tokens rather than letters, tasks like counting characters can trip them up. So for ad headlines and SMS, always check the count yourself or with a counter tool.

2:24 The context window

Next, the context window. Think of it as the model's whiteboard, or working memory. Everything it can consider right now goes on that whiteboard: your instructions, any files you pasted, and the conversation so far. Modern assistants have very large whiteboards, big enough for long documents. But they're still finite. And even before the board is full, in a very long chat with lots of detours, the model can give less attention to instructions from way back at the start. That's why your brand rules seem to fade in hour two. The model isn't being lazy. The whiteboard is crowded.

3:07 Training and inference

Now training and inference. Training is how the model learned, from a huge amount of data up to a cut off date. It's like a library that stopped acquiring books on a certain day. Anything after that date, the model simply doesn't know, unless you give it search or paste in the information. Inference is the model at work: generating an answer, right now, from its training plus whatever's on the whiteboard. One more thing about inference. There's a bit of randomness in how the model picks each next token. That's why asking the same question twice can give you two different answers, and why two different numbers for the same fact is a strong hint that it doesn't really know.

4:00 Example 1: the outdated policy

Here's a simple example. An influencer manager asks an assistant, with search turned off, about a platform's current rules for labeling paid partnerships. The answer is confident, clear, and out of date, because the platform updated its rules after the model's training cut off. Nothing was invented, exactly. It's just old. The fix is to give the model today's facts. Turn on web search and ask for the official source, or paste the current policy text and say: answer only from this text. Now the answer quotes the current rule. Rule of thumb: for anything that changes, like policies, prices, platform features and trends, never rely on training alone.

4:47 Example 2: the three-hour campaign chat

Now a realistic business case. Hamza, a marketing lead at an electronics retailer in Riyadh, spends an afternoon planning a back to school campaign in one enormous chat. At the start, he sets rules: no discount language, because the brand is positioned on quality, and always mention the two year warranty. By hour two, drafts start saying huge savings, and the warranty disappears. The whiteboard is crowded. Hamza asks for a one hundred and fifty word summary of the goal, audience, brand rules, decisions and open tasks, opens a fresh chat, pastes it, and continues. The next drafts follow the rules again. Then he moves the brand rules into his project instructions, so they apply to every chat from now on.

5:40 Watch me do it

Let me show you the rescue. I'm deep into a long chat about a product launch, and I notice the latest draft uses exclamation marks everywhere, which I banned at the start, and it's calling the product by its old name. That's drift. So I type: summarize everything important from this conversation in under one hundred and fifty words: the goal, the audience, the brand rules, the decisions made and the open tasks. It gives me a tight summary. I check it, fix one detail, the product's new name, and copy it. Then I open a new chat inside my project, paste the summary, and ask for the next draft. Clean. Finally, I open the project instructions and add the two rules that drifted, so they're applied every time.

6:36 Reasoning modes and limits

A quick word on reasoning modes and limits, since you'll see them in most assistants. Reasoning or thinking modes let the model work through a problem step by step before it answers. That uses more tokens and more time, so it's worth it for analysis, planning and tricky questions, like comparing three pricing options, but it's unnecessary for a caption or a hashtag list. And your plan's limits are usually measured in tokens. Very long files, very long chats, and text in scripts that use more tokens per word all use them faster. Match the mode to the job.

7:19 Recap and try this now

Let's recap. Tokens are how models read and count, so check character limits yourself. The context window is the model's working memory, so when a long chat drifts, restart it with a summary, and keep stable rules in project instructions or custom instructions. Training has a cut off, so give the model today's facts with search or pasted sources. And inference has some randomness, so if the same question gets different numbers, treat that as a sign the model doesn't know. Here's your try this now. Find your longest running AI chat. Ask for the one hundred and fifty word summary, start fresh with it, and move your brand rules into a project. Compare the next draft with the last one.