Embeddings and semantic search

0:00 Why search misses the obvious

Here's a puzzle. A customer types, the app keeps logging me out. Your help centre has the perfect article, titled Session timeout settings. Old-fashioned keyword search returns nothing useful, because the two sentences share no important words. By the end of this lesson you'll understand exactly why that happens, how embeddings fix it, where embeddings fail, and how to combine both approaches so your search, and later your AI assistant, finds the right answer far more often.

0:33 Why it matters

Why should you care? Because almost every AI feature you'll build in this course starts with finding the right information. A support assistant, a sales research agent, a policy chatbot: if search fails, everything downstream fails, however clever the model. Here's an analogy. Keyword search is like a librarian who only matches the exact words on the spine. Semantic search is like a librarian who understands what you mean, even if you describe the book badly. The best libraries employ both.

1:08 Embeddings = meaning as coordinates

An embedding is a list of numbers that represents meaning. An embedding model has been trained on huge amounts of text so that things which mean similar things end up close together in a space with hundreds or thousands of dimensions. You can't picture that many dimensions, so imagine a map instead. How do I get a refund, and can I send my order back for my money, land in the same neighbourhood. Refund and refinery share letters, but they live in different cities. Search then becomes geography: turn the question into a point on the map, and look for the nearest documents.

1:53 Measuring similarity

How do we measure near? The usual answer is cosine similarity, which compares the direction of two vectors. If you normalise your vectors, a plain dot product gives the same ranking, which is why many libraries do exactly that. At small scale you can compare the question with every document. At millions of documents, systems use approximate nearest neighbour indexes, such as HNSW graphs, trading a sliver of accuracy for a big jump in speed. For most business knowledge bases, a vector extension on a database you already run, like pgvector on Postgres, is plenty.

2:34 Simple example

Let's make it concrete with a simple example. Take three sentences. One: my card was declined. Two: the payment didn't go through. Three: the card game was fun. An embedding model places one and two close together, because they mean nearly the same thing, and places three far away, even though it shares the word card. Now ask, why did my payment fail? The nearest neighbours are one and two. That's semantic search in a single picture, and it's exactly why it rescues paraphrased questions.

3:11 Keyword vs semantic vs hybrid

Now the catch. Semantic search is brilliant at paraphrases and natural questions, but weak at exact things. Ask for error E1042 and it may return general articles about errors, because they all feel similar. Keyword search, the BM25 family of ranking, is the opposite: great for product codes, names, clause numbers and error messages, useless for paraphrases. That's why hybrid search is the strong default for business content. You run both searches and fuse the ranked lists. A popular method is reciprocal rank fusion, which rewards documents that rank highly in either list and needs no fiddly score calibration.

3:54 Choosing an embedding model

Choosing an embedding model is where teams waste the most time reading leaderboards. Leaderboards are a starting point, not a verdict. What matters is how a model performs on your questions, in your users' languages, with your jargon. If your customers write in English, Arabic, Urdu and Roman Urdu, test all four. Remember three practical rules. Bigger vectors cost more to store and search. Some models let you shorten vectors with modest quality loss. And if you ever switch models, you must re-embed everything, because vectors from different models are not comparable.

4:34 Metadata filters matter

There's one more lever that often beats a fancier model: metadata. Store the source, date, product, region and access level with every chunk, and filter on it. A Dubai employee asking about leave should only see the Dubai policy, and only documents they're allowed to open. Filtering improves relevance and enforces permissions at the same time. Without it, your AI assistant can become a very polite way for people to read documents they should never have seen.

5:07 Business example (illustrative)

Let's deepen the business picture with an illustrative case. A Riyadh electronics retailer has around two thousand help articles in Arabic and English. Before hybrid search, its logs showed roughly one search in four ending with no click. After adding semantic search fused with keyword search, plus a filter for the customer's country, that dropped to about one in ten, and searches for exact model numbers still worked. The numbers are illustrative, but the pattern is typical: hybrid fixes paraphrases without breaking codes.

5:43 Hands-on in the lesson

In the lesson text you'll find a hands-on script that runs on a laptop with no API key. It builds keyword search, semantic search and fused hybrid search over a handful of help articles, including one in Arabic, then measures recall at three, which simply asks: for what share of questions did the right document appear in the top three results? Run it, then replace the sample articles with twenty of your own and twenty real questions. That tiny benchmark will teach you more about your data than any vendor demo.

6:23 Common mistakes

Before we recap, the most common mistakes. First, testing on questions you wrote yourself, which mirror your documents' wording and make any search look good. Second, switching the query embedding model without re-embedding the documents. Third, forgetting metadata filters, so users retrieve documents from the wrong region or ones they shouldn't see. Fourth, ignoring languages: an English-only model quietly failing your Arabic or Urdu speakers. And fifth, choosing a vector database before measuring anything. Measure first, then choose infrastructure.

6:57 How you'll know it's working

How will you know your search is working? Track three numbers. Recall at three or five on your labelled questions, which should rise as you move from keyword to hybrid. The share of real user searches that end in a click or a solved question. And the no-results rate, which should fall. If recall rises but users still can't find things, your test questions don't reflect real behaviour, so add more real queries from your logs.

7:30 Watch me do it: hybrid search script

Watch me do it. I open the script from the lesson. First, the four documents, one of them in Arabic. Next, I load the multilingual model and call encode with normalise embeddings set to true, so a dot product equals cosine similarity. Then I build the BM25 index by splitting each document into lower-case words. The semantic function embeds the query and sorts documents by score. The keyword function asks BM25 for scores. And rrf adds one over sixty plus rank for every list a document appears in. I run it. For the app keeps logging me out, keyword search misses, semantic search finds kb-01, and hybrid keeps it on top. For E1042, keyword wins and hybrid still ranks kb-02 first. For can I get my money back, both refund articles appear, including the Arabic one. Finally, I run recall at k on the three labelled questions and see hybrid score highest. That's the whole loop: build, run, measure.

8:39 Recap

Let's recap. Embeddings turn meaning into coordinates, and semantic search finds the nearest ones. Keyword search wins on exact terms, semantic wins on paraphrases and languages, and hybrid with rank fusion gives you both. Choose models by testing on your own questions, store and filter on metadata, and re-embed everything if you change models. Your next step: collect twenty real questions from your inbox or help desk, label the right answer for each, and measure. Next, we'll look at chunking, the quiet decision that controls what your system can find in the first place.

9:20 Try this now (20 minutes)

Try this now. Open your support inbox, help desk or sales enquiries and copy ten real questions. For each one, note which article or document should answer it. Then run the hands-on script, replacing the sample articles with those documents, and record how many questions get the right answer in the top three for keyword, semantic and hybrid search. It takes about twenty minutes, and you'll have your first retrieval benchmark, which you'll reuse in every lesson that follows.