Voice AI & Conversational AgentsUse cases and capstone · Lesson 17 of 17
Capstone: build an appointment-booking voice agent
Video lecture
Capstone: build an appointment-booking voice agent
The narrated lecture is in production
Every chapter is scripted and ready. Browse the chapters and read the full transcript now — the video will appear here when it’s published.
Chapters
Transcript of the narration, chapter by chapter.
0:00 Capstone: booking voice agent
It's time to put everything together. In this capstone, you'll build an appointment-booking voice agent that answers the phone, books, reschedules and cancels appointments in a real or mock calendar, hands off to humans, and meets the disclosure, privacy and quality standards from this course. Use your own business or a client, or our default case, Nova Dental, with branches in Dubai Marina, Downtown Dubai and London Soho, and callers in English and Arabic.
0:32 Why this capstone
Why a booking agent as the capstone? Because it exercises every skill from the course in one realistic project: conversation design, prompts, tools with business rules, telephony, multilingual callers, disclosure and consent, testing and operations. And it's directly useful: clinics, salons, service centers and agencies everywhere need exactly this, so your capstone can become a real product or client project.
0:58 Ten deliverables
You'll produce ten deliverables, most of which you've practiced already. A conversation design spec with three sample dialogues. A voice-style system prompt and your ten-utterance test results. Tools for availability, booking, rescheduling and canceling, plus transfer and end call. A bilingual FAQ-style knowledge base. Your voice choice and pronunciation dictionary. Telephony or widget deployment. A compliance pack with disclosure lines in English and Arabic, privacy text and retention settings. A test suite with a launch gate. A monitoring plan and unit economics. And a three-minute demo with a one-page launch memo.
1:38 Analogy: opening a front desk
An analogy for the capstone: you're opening a new front desk, not writing a chatbot. A real front desk has a trained receptionist, a booking system with rules, a phone line, a manager to escalate to, notices on the wall, a training checklist, and someone reviewing how it's going. Every deliverable in this capstone maps to one of those. If your front desk would be ready for real patients on Monday, your capstone is done.
2:11 Design + backend
Start with design and the backend. Required slots are service, branch, date and time, name and mobile, with explicit confirmation before booking, and a repair path after two failures. Then build a booking API with endpoints for availability, book, reschedule and cancel. Enforce the rules on the server: no past dates, no double bookings, valid branches, and an identity check using mobile number and name before rescheduling or canceling. Protect it with a shared secret header, make booking idempotent so retries don't create duplicates, and return say messages the agent can use when something fails.
2:52 Agent + tests
Next, configure the agent. Use the voice prompt template, a low-latency voice, sensible turn settings, your tools and knowledge base, language detection, transfer to reception and end call. Add evaluation criteria for booked or changed, disclosed AI in the first turn, and no clinical advice, plus data collection for service, branch, outcome and language. Then write tests: at least ten simulations, including Arabic, a date change, a double-booking race, an angry caller and an emergency; eight tool-call tests; six safety tests; and ten audio clips with noise and accents. Define your launch gate before you run them.
3:34 Deploy + operate
Deploy to a test number through a Twilio import or SIP trunk, or a web widget if you can't get a number. Test on real mobiles, in noise, and test transfers both during and outside business hours. Then set up operations: a dashboard with alerts for disclosure failures, p90 latency and success drops, a weekly review ritual with an owner, the unit economics sheet, and a rollback plan. Plan a staged rollout, for example twenty percent of calls in week one, then everything if the metrics hold.
4:12 Launch memo + demo
Finish with a launch memo, one page: scope and what's out of scope, launch gate results against targets, top risks and mitigations, the compliance summary, the monitoring plan, the rollout and rollback plan, and the decision you need from leadership. Then record a three-minute demo showing a real call: a booking, a correction, a transfer, and a polite refusal of clinical advice. Judge your work with the rubric in the lesson text: conversation design, tool safety, compliance, testing, operations and the demo.
4:48 Stretch goals
If you want to go further, try three stretch goals. Add outbound reminder calls the day before, with consent checks, calling windows and voicemail detection. Add WhatsApp or text confirmations with reschedule links. Or build a speech to speech version with OpenAI Realtime or Gemini Live, and compare it with your cascade on latency, success rate and cost per booking. That comparison is a genuinely useful piece of work for any business deciding which architecture to bet on.
5:22 Common mistakes
Common capstone mistakes. Putting business rules like no double bookings only in the prompt, instead of the API. Skipping idempotency, so a retry creates two bookings. Writing disclosure text in English only for bilingual callers. Defining the launch gate after seeing the test results, which makes it meaningless. And demoing only the happy path, when the real proof is a correction, a transfer and a polite refusal.
5:51 Watch me do it: the full demo call
Watch me do it. I show the finished Nova Dental agent working end to end. On the left, the config file with every section filled: architecture decision, voice, turn taking, tools, knowledge base, compliance, tests and launch gate. On the right, a live call from my mobile. Aria answers with the AI disclosure and recording notice. I ask for a cleaning at Dubai Marina next Thursday afternoon. She says let me check, calls availability, and offers two times. I pick two thirty, then correct myself: actually, make it Tuesday. She accepts the change without fuss, checks again and offers Tuesday times. She reads back the service, date, time, branch and my name, and I say yes. The booking tool runs with an idempotency key, and I receive a text confirmation. Then I ask: is this pain in my back tooth serious? She gives the approved wording, offers the earliest appointment, and names emergency services for severe symptoms. Finally I ask for a person, and she transfers me to reception with a summary on the screen. After the call, the post-call webhook shows the evaluation results: booked, disclosure given, no clinical advice. The launch gate is green, the rollout is set to twenty percent, and the launch memo is ready for sign-off.
7:23 Course recap
That's the course. You now understand how voice agents work, how to manage latency and turn-taking, how to build with ElevenAgents, realtime APIs and open-source stacks, how to connect telephony, design conversations and prompts, handle tools and knowledge, serve Urdu, Arabic and English callers, respect consent and the law, test rigorously and operate at scale. Build the capstone, ship it carefully, and keep improving it every week. Your callers will notice.
7:54 Try this now
Try this now. Start with the backend, not the agent. Build the four booking endpoints with rules enforced on the server and an idempotency key on booking. Then write your launch gate and at least five simulation tests before you configure the agent. Only then set up the agent, connect a test number or widget, and run the tests. Record your demo only after the launch gate passes.
The brief
Build, test and launch-ready an appointment-booking voice agent for a real or realistic business: a clinic, salon, car service center, tutoring center or property viewing desk. It must answer the phone (or a web widget if you cannot provision a number), book, reschedule and cancel appointments in a real or mock calendar, hand off to humans, and meet the disclosure, consent, privacy and quality standards from this course.
Default case: "Nova Dental" with branches in Dubai Marina, Downtown Dubai and London Soho; English and Arabic callers; services: check-up, cleaning, whitening consultation, emergency slot.
Deliverables
| # | Deliverable | From lesson |
|---|---|---|
| 1 | Conversation design spec + three sample dialogues | 4.1 |
| 2 | System prompt (voice template) + ten-utterance test script results | 4.2 |
| 3 | Tools: availability, book, reschedule, cancel (webhooks) + transfer + end call | 2.2, 4.3 |
| 4 | Knowledge base: services, hours, prices, policies (FAQ-style, both languages) | 4.3, 4.4 |
| 5 | Voice choice with test notes; pronunciation dictionary | 2.1 |
| 6 | Telephony or widget deployment; transfer path | 3.1 |
| 7 | Compliance pack: AI + recording disclosure lines (EN/AR), privacy notice text, retention settings, consent approach for reminders | 5.1, 5.2 |
| 8 | Test suite: simulations, tool-call tests, safety tests, audio clips; launch gate | 6.1 |
| 9 | Monitoring plan + unit economics sheet | 6.2 |
| 10 | 3-minute demo recording and a one-page launch memo | This lesson |
Build plan (suggested 8 to 12 hours)
1. Design (1.5 h). Required slots: service, branch, date/time, name, mobile. Explicit confirmation before booking. Repair after two failures: keypad or human. Emergency wording approved.
2. Backend (2 h). Build a mock booking API (FastAPI, a Google Calendar or Microsoft 365 calendar integration, or your practice-management system's API). Endpoints: /availability, /book, /reschedule, /cancel. Enforce rules server-side: no past dates, no double booking, valid branches, identity check (mobile number + name) for reschedule/cancel. Protect with a shared secret header.
# Minimal booking rules sketch (extend the availability webhook from Lesson 4.3)
@app.post("/book")
def book(body: BookIn, x_tool_secret: str = Header(default="")):
require_secret(x_tool_secret)
slot = get_slot(body.branch, body.start_iso)
if slot is None or slot.taken:
return {"ok": False, "say": "That time was just taken. Offer the next two available times."}
booking = create_booking(slot, body.name, body.mobile, body.service) # idempotency key = mobile+start_iso
send_sms_confirmation(body.mobile, booking) # optional
return {"ok": True, "booking_id": booking.id,
"say": f"Booked {body.service} on {booking.spoken_time} at {booking.spoken_branch}."}3. Agent (2 h). Configure on ElevenAgents (or your chosen stack): prompt from the voice template; Flash-class TTS voice; turn settings; tools; knowledge base; language detection; transfer to reception; end call; evaluation criteria (booked_or_changed, disclosed_ai_first_turn, no_clinical_advice); data collection (service, branch, outcome, language).
4. Tests (2 h). At least: 10 simulations (including Arabic, a date change, a double-booking race, an angry caller, an emergency), 8 tool-call tests, 6 safety tests (clinical advice, invented price, injection, disclosure, recording notice, opt-out), 10 audio clips with noise and accents. Define your launch gate.
5. Deploy (1 h). Connect a test number via Twilio import or SIP (or a web widget). Test on real mobiles. Test transfer during and outside hours.
6. Operate (1 h). Dashboard and alerts; weekly review ritual; unit economics sheet; rollback plan.
Launch memo template
# Launch memo: Nova Dental AI receptionist (v1.0)
Scope: inbound bookings, reschedules, cancellations; EN/AR; 3 branches. Not in scope: clinical triage, payments.
Launch gate results: disclosure 100% | safety 0 failures | simulation success X% (target Y%) |
median/p90 latency A/B ms (budget C) | Arabic success D% (min E%)
Risks and mitigations: [top 3]
Compliance: AI + recording disclosure (EN/AR approved text); privacy notice updated; retention 60 days;
no outbound calls in v1; data processing locations confirmed with vendor.
Monitoring: alerts on disclosure failure, p90 latency, success drop; weekly review owner: [name]
Rollout: 20% of calls for week 1 via traffic split -> 100% if metrics hold. Rollback: previous version.
Decision requested: approve staged launch on [date].Assessment rubric (self or peer)
| Criterion | Excellent |
|---|---|
| Conversation design | Natural, short turns; smart confirmations; good repair; bilingual |
| Tool safety | Rules enforced server-side; honest failures; idempotent booking |
| Compliance | Disclosure and recording notice in first turn; approved translations; retention set; no improvised legal text |
| Testing | Launch gate defined and met; safety suite passes; per-language metrics |
| Operations | Alerts, weekly loop, staged rollout, rollback, unit economics |
| Demo | A real call showing booking, a correction, a transfer and a refusal of clinical advice |
Stretch goals
- Outbound reminder calls the day before, with consent checks and voicemail detection.
- WhatsApp or SMS confirmations with reschedule links.
- A speech-to-speech variant (OpenAI Realtime or Gemini Live) compared against your cascade on latency, success and cost.
Key takeaways
- The capstone agent books, reschedules and cancels via server-enforced, idempotent tools with identity checks and honest failure messages.
- It discloses AI and recording in the first turn with approved English and Arabic text, and never gives clinical advice.
- A launch gate (disclosure, safety, success, latency, per-language minimums) must pass before a staged rollout with rollback.
- A one-page launch memo and a real demo call communicate scope, results, risks, compliance and the decision needed.
Check your understanding
Quick questions to lock in the lesson. They don’t count towards your certificate.
Put it into practice
Build the Nova Dental (or your own) booking agent: backend, agent config, tests with a launch gate, deployment, monitoring and a launch memo. Record a 3-minute demo call covering a booking, a correction, a transfer and a refusal of clinical advice.
Enrol for free to save your progress
Reading is always free. Enrol to keep your place, take the final assessment and earn a verifiable certificate.