Skip to content

AI Automation & Agents for Small Business · AI assistants and agents · lesson 8 of 16 · 13 min

Knowledge bases that keep agents accurate

Your agent is only as good as what it knows

Most embarrassing agent answers are not model failures; they are knowledge failures: an old price list still in the knowledge base, a policy written ambiguously, two documents that contradict each other, or a question the documents never answer (so the agent improvises). Behind most no-code agents' "knowledge" features is retrieval-augmented generation (RAG): documents are split into chunks, indexed, and the most relevant chunks are given to the model when a question arrives. You do not need to build that machinery, but you do need to feed it well.

Principles for agent-ready knowledge

  1. One topic per section, with a clear heading. Retrieval finds sections; a heading like "Refund policy for cancelled shoots" is findable, "Other info" is not.
  2. Question-and-answer format for FAQs. Write the question the way customers actually ask it, then a complete, self-contained answer.
  3. Self-contained answers. Avoid "see above" or "as mentioned". Each chunk may be read alone.
  4. Explicit numbers, dates and conditions. "Rescheduling is free up to 48 hours before the shoot; after that a 20% fee applies." Not "reasonable notice".
  5. State what you don't do. "We do not offer drone photography." Agents otherwise improvise yes.
  6. One source of truth. Remove superseded documents; do not keep "prices-2025-final-v2" beside "prices-2026".
  7. Owner and review date on every document. Knowledge rots; someone must own it.
  8. Languages. If customers ask in Arabic or Urdu, test retrieval in those languages; consider parallel versions of key documents.
  9. No sensitive data. Internal notes, staff phone numbers or client names do not belong in a customer-facing knowledge base.

Where knowledge lives in common tools

  • Assistant products (custom GPTs, Claude Projects, Copilot agents): upload files or connect drives; the platform handles retrieval.
  • No-code builders: n8n offers vector store nodes (from a simple in-memory store for testing to hosted vector databases) and document loaders; Zapier and Make agents can connect to knowledge sources.
  • Voice and chat agent platforms (for example ElevenLabs Agents): a knowledge-base section where you add documents or URLs; some can report which sources they used.

Whatever the tool, the steps are the same: prepare, load, test, maintain.

Worked example: a clinic-marketing agency's FAQ rebuild

An agency in Karachi runs an after-hours enquiry agent for a dental clinic. Early transcripts show the agent quoting an old whitening price and telling one visitor the clinic opens on Sundays (it does not). Investigation: the knowledge base held three versions of the price list and a generic "About us" page that mentioned "open 7 days" from an old promotion.

The fix: one current services-and-prices.md with an owner and review date, a rewritten FAQ in Q&A format (English and Urdu), an explicit "What we don't do" section (no medical advice by chat, no emergency service), and removal of all old files. They wrote 25 test questions from real transcripts and re-ran them after every knowledge change. Wrong-answer reports dropped to near zero over the next month.

Hands-on: prepare, test and maintain a knowledge base

1. Convert messy documents into agent-ready Q&A with an assistant (use non-sensitive documents only):

Rewrite the document below as an FAQ for a customer-facing AI assistant.
Rules: one question per heading, written the way a customer would ask it; complete, self-contained
answers; keep every number, date and condition exactly; if two statements conflict, list the conflict
at the end instead of choosing; add a "What we don't offer" section listing anything explicitly excluded;
do not invent information. Output Markdown.
<document>{{PASTE}}</document>

Then have the document owner check every answer before loading it.

2. Use this template for each knowledge document:

# Services and prices — Lens & Light Studio
Owner: Studio manager | Last reviewed: 2026-09-01 | Next review: 2026-10-01 | Supersedes: prices-2025.pdf

## How much does a family shoot cost?
A family shoot (90 minutes, up to 6 people) costs AED 1,400-1,800 depending on location...

## Can I reschedule?
Rescheduling is free up to 48 hours before the shoot. After that, a 20% fee applies...

## What we don't offer
- Drone photography
- Same-day delivery of edited photos

3. Test retrieval, not just answers. Build 20 to 30 questions from real enquiries (including Arabic, Urdu and misspellings) and at least five questions the knowledge base should not answer. For each, record whether the agent answered correctly, cited or used the right document (if your tool shows sources), or correctly handed off.

4. Maintain it. A monthly 20-minute review: owner checks prices and policies, removes superseded files, adds answers for new questions seen in transcripts, and re-runs the test questions.

Video lecture: Knowledge bases that keep agents accurate

Lecture coming soon · 15 chapters · about 8 minutes. Read the full transcript below.

  1. Knowledge bases for agents
  2. Analogy: the briefing folder
  3. How agent knowledge works
  4. Principles 1–5
  5. Principles 6–9
  6. Simple example: 'usually quick'
  7. Worked example: dental clinic agent
  8. Business example (illustrative)
  9. Hands-on in the lesson
  10. Maintain monthly
  11. Common mistakes
  12. How you'll know it's healthy
  13. Watch me do it: rewrite and test
  14. Recap
  15. Try this now (45 minutes)

Lecture transcript

Knowledge bases for agents

When an agent tells a customer the wrong price, most people blame the AI. Usually, the AI did exactly what it was told. The old price list was still in the knowledge base. Or the policy said reasonable notice instead of forty-eight hours. Or two documents disagreed. In this lesson you'll learn how agent knowledge works behind the scenes, nine principles for writing knowledge an agent can use, how to test it, and how to keep it from rotting.

Analogy: the briefing folder

Here's an analogy. A knowledge base is your agent's briefing folder. If the folder contains last year's price list, a half-finished policy and a leaflet from an old promotion, even the best employee will say the wrong thing. If it contains one current, clearly organised document per topic, with the date on it, a new hire can answer confidently on day one. Agents are the same, only faster and more literal.

How agent knowledge works

Behind most agent knowledge features is retrieval-augmented generation. Your documents are split into chunks and indexed. When a question arrives, the most relevant chunks are handed to the model, which answers from them. You don't need to build that machinery, because assistant products, no-code builders and voice platforms do it for you. But you do need to feed it well, because retrieval can only find what's clearly written, and the model can only be as current as the chunks it gets.

Principles 1–5

Principles one to five. One topic per section with a clear heading, because retrieval finds sections, and refund policy for cancelled shoots is findable while other info isn't. Question-and-answer format for FAQs, written the way customers actually ask. Self-contained answers, never see above, because each chunk may be read alone. Explicit numbers, dates and conditions: free rescheduling up to forty-eight hours before, then a twenty percent fee. And state what you don't do, like no drone photography, because agents otherwise improvise a yes.

Principles 6–9

Principles six to nine. One source of truth: remove superseded files. An owner and review date on every document, because knowledge rots and someone has to own it. Languages: if customers ask in Arabic or Urdu, test retrieval in those languages, and consider parallel versions of key documents. And no sensitive data: internal notes, staff phone numbers and client names don't belong in a customer-facing knowledge base.

Simple example: 'usually quick'

A simple example. Your FAQ says: delivery is usually quick. A customer asks, will it arrive by Friday if I order today in Jeddah? The agent has nothing concrete, so it either guesses or hedges unhelpfully. Rewrite the entry: orders placed before two p.m. Sunday to Thursday are delivered in two to three working days within Riyadh and Jeddah, and three to five elsewhere in Saudi Arabia. Now the agent can answer precisely, and so can your staff.

Worked example: dental clinic agent

Here's a worked example. A Karachi agency runs an after-hours enquiry agent for a dental clinic. Transcripts show it quoting an old whitening price and telling someone the clinic opens on Sundays. It doesn't. The knowledge base held three versions of the price list, and an old About us page said open seven days from a past promotion. The fix: one current services and prices document with an owner and review date, a rewritten FAQ in English and Urdu, a what-we-don't-do section covering no medical advice by chat and no emergency service, and every old file removed. Twenty-five test questions from real transcripts are re-run after every change, and wrong-answer reports drop to near zero.

Business example (illustrative)

Illustrative numbers for the dental clinic agency. Before the rebuild, about one after-hours answer in eight was flagged wrong by staff reviewing transcripts. After consolidating to one owned price document, rewriting the FAQ in English and Urdu, and adding the exclusions section, that fell to about one in sixty over the next month. The twenty-five-question test set now runs after every change, taking about ten minutes.

Hands-on in the lesson

In the hands-on section, you'll use a prompt that rewrites messy documents into agent-ready Q&A, keeping every number and date, flagging conflicts instead of choosing, and adding an exclusions section, never inventing anything. The document owner then checks every answer. You'll get a document template with owner, review date and a supersedes line. And a testing approach that checks retrieval as well as answers, with twenty to thirty real questions including other languages, misspellings and at least five questions it should not answer.

Maintain monthly

Then maintain it. Schedule a twenty-minute monthly review. The owner checks prices and policies, removes superseded files, adds answers for new questions seen in transcripts, and re-runs the test questions. Put it in the calendar, because knowledge that nobody owns will be wrong within a quarter, and your agent will confidently repeat whatever it finds.

Common mistakes

Common mistakes. Uploading a whole shared drive and hoping for the best. PDFs of scanned brochures the tool can barely read. Answers that depend on another section, like see above. Internal notes and staff contacts left in customer-facing documents. No owner or review date. And never testing with questions the knowledge base shouldn't answer, so improvisation goes unnoticed.

How you'll know it's healthy

How will you know your knowledge base is healthy? Your test questions pass, including the ones in Arabic, Urdu or Roman Urdu. The agent hands off correctly on questions the documents don't cover. Wrong-answer reports are rare. Every document has a current review date. And new questions from transcripts become new FAQ entries within a month.

Watch me do it: rewrite and test

Watch me do it. I take the studio's old services brochure and paste it into the rewrite prompt. The output has one customer-phrased question per heading: how much does a family shoot cost, can I reschedule, do you travel outside Dubai. Every number is kept exactly, and at the end it lists one conflict: the brochure says forty-eight hours for rescheduling on one page and twenty-four on another. I ask the studio manager, who confirms forty-eight, and I fix it. Next, I add the header: owner, last reviewed, next review and supersedes. I add a what we don't offer section with drone photography and same-day delivery. Then I load the document into the agent's knowledge base and remove the old brochure. Finally, I run twenty questions. Eighteen pass. Can you do a drone shoot correctly says no. Do you do corporate headshots gets an improvised yes, so I add corporate headshots to the document and re-run.

Recap

To recap: most bad agent answers are knowledge failures. Write agent-ready knowledge with clear headings, Q&A, self-contained answers, explicit numbers and exclusions. Keep one owned source of truth, keep sensitive data out, and test in your customers' languages with questions it shouldn't answer. Your next step: rewrite one customer-facing document into agent-ready Q&A and test it with twenty real questions. Next module: human-in-the-loop playbooks.

Try this now (45 minutes)

Try this now. Pick one customer-facing document, like your prices, delivery policy or services list. Run it through the rewrite prompt, then check every answer yourself. Add an owner, a review date and a what-we-don't-offer section. Load it into your agent or assistant and test it with twenty real questions, including five it shouldn't answer. Note every miss and fix the document, not the prompt.

Key takeaways

  • Most bad agent answers are knowledge failures: stale, conflicting, vague or missing information.
  • Write agent-ready knowledge: clear headings, Q&A format, self-contained answers, explicit numbers and a “what we don’t offer” section.
  • Keep one source of truth with an owner and review date; remove superseded files and keep sensitive data out.
  • Test with real questions (including other languages and unanswerable ones) and re-run after every change.

Try it

Rewrite one customer-facing document into agent-ready Q&A with an owner and review date, then test it with 20 real questions including five it should not answer.