Voice AI & Conversational AgentsTelephony and deployment · Lesson 6 of 17
Telephony: phone numbers, SIP trunks and Twilio
Video lecture
Telephony: phone numbers, SIP trunks and Twilio
The narrated lecture is in production
Every chapter is scripted and ready. Browse the chapters and read the full transcript now — the video will appear here when it’s published.
Chapters
Transcript of the narration, chapter by chapter.
0:00 Telephony for voice agents
Here's something that surprises web developers: a voice agent that sounds brilliant in a browser can sound thin, laggy or confused on a real phone call. Phone calls cross a different world, with narrowband audio, codecs, carriers, caller I D rules and regulations. Since most business voice agents live on the phone, this lesson teaches you the plumbing: numbers, SIP trunks, Twilio, inbound and outbound flows, transfers, voicemail and keypad input.
0:31 Why telephony
Why does telephony matter if the agent works in the browser? Because most business callers still pick up a phone. And the phone network changes things: narrower audio, carrier routing, caller ID, regulations about automated calls, and the need to transfer to humans reliably. Understanding the plumbing is what turns a demo into a phone line your customers can actually rely on.
0:58 Key terms
A few terms first. A D I D is simply a phone number people dial, bought from a carrier or a communications platform like Twilio, Vonage, Telnyx or Plivo. SIP, the Session Initiation Protocol, is the signaling standard that sets up, transfers and ends internet calls, while the audio itself travels separately as media. A SIP trunk connects your phone system or carrier to another SIP endpoint, like your agent platform. Phone networks often use narrowband codecs. D T M F means keypad tones. And a SIP REFER is one way to transfer a call.
1:39 Analogy: posting a letter
An analogy for telephony: think of a phone call as a letter posted through several post offices. The phone number is the address. The carrier and your communications platform are the post offices that route it. SIP is the envelope and the delivery instructions. The audio itself is the letter inside, carried separately. And a transfer is forwarding the letter to another address, ideally with a sticky note explaining what it's about, so the new recipient doesn't have to ask again.
2:14 Three ways to connect
There are three ways to connect an agent to phones. First, native platform integration: import a Twilio number into a platform like ElevenAgents and route calls to your agent, with outbound calls through the API. Second, SIP trunking: point your existing phone system, contact center or local carrier at the agent's SIP endpoint. That's ideal if you want to keep your UAE or Pakistan numbers with a local carrier. Third, media streaming: Twilio Media Streams send raw call audio to your server over WebSocket, and ConversationRelay handles speech for you and sends text to your app. More code, more control.
2:57 Inbound flow
Follow an inbound call. The caller dials your number. The carrier routes it to your communications platform, which forwards it by webhook or SIP to your agent. The agent greets them, disclosing that it's an AI and, if you record, saying so. It looks the caller up in your CRM by caller I D, uses booking tools, and either resolves the request or transfers to a human with context. When the call ends, a post-call webhook sends the transcript, evaluation results and extracted data to your systems.
3:35 Outbound calls
Outbound calls, like reminders and lead follow-ups, carry the heaviest legal risk, which we'll cover in module five. Technically, you load the contact and their consent status from the CRM, pass dynamic variables like name and appointment time at the start of the call, and enable voicemail detection so the agent leaves a short message or hangs up. Retry politely, with daily limits and respect for the recipient's time zone. And log outcomes back to the CRM. For campaigns, batch calling needs pacing and suppression lists.
4:12 Transfers and audio
Transfers make or break the experience. Tell the caller what's happening: I'm connecting you to Sara at reception now. Pass context, like a summary, the caller's details and the reason, through SIP headers or to the human's screen, so nobody repeats themselves. Have a fallback when nobody answers, like taking a callback number. And respect business hours. Don't transfer to an empty line at midnight. On audio quality: test on real phones, use noise suppression, and let callers type long numbers on the keypad when speech struggles.
4:50 Worked example: Riyadh service center
A car service center in Riyadh keeps its Saudi number with a local carrier and connects it by SIP trunk to the agent platform. The agent answers in Arabic by default, detects English speakers, books service slots through the dealership system's API, and during working hours transfers to the service desk with a summary. For outbound, it calls customers who opted in on their service contract the day before their appointment, and voicemail detection leaves a short Arabic message. Every call opens with an AI disclosure, and they checked where the vendor processes data against Saudi transfer rules.
5:33 Example 2: Birmingham plumber
A simple example. A plumber in Birmingham wants after-hours calls answered. He keeps his existing number and sets call forwarding after six p m to a new Twilio number imported into his agent platform. The agent takes the address, the problem and urgency, texts him a summary for emergencies, and books non-urgent visits into his calendar. No SIP trunk, no media streams, no code: native integration is enough for a single-number business.
6:04 Common mistakes
Common telephony mistakes. Testing only on laptop speakers, when phone audio is narrowband and behaves differently. Cold transfers with no context, so callers repeat everything to the human. Transferring to an empty line outside business hours. And outbound campaigns without consent records, calling-hour logic or suppression lists, which is where the biggest legal and reputational risks sit.
6:29 Watch me do it: connect a real number
Watch me do it. I connect Aria to a real phone number. In the telephony provider's console I buy a test number with voice capability. In the agent platform I go to phone numbers, choose import from Twilio, and paste the account S I D, the auth token from my secret store, and the number. I assign the number to the Aria agent. Then I call it from my own mobile, standing in a noisy corridor. The agent answers with the disclosure and recording notice. I ask for a Thursday appointment and say my mobile number in groups; the agent reads it back correctly. Next I test a transfer: I say, can I speak to someone? The agent says it's connecting me to reception, and my second phone rings, with a summary shown in the platform's call details. Then I test outbound, using the Python function from the lesson text. I set a test contact with consent set to true and my number, run it, and my phone rings with a reminder using my name and appointment time from the dynamic variables. I let the next test go to voicemail, and the agent leaves the short message instead of chatting to a machine. Finally, I set consent to false and run it again: the function refuses to dial, and logs the reason.
8:06 Recap and next step
Recap. Phone calls are narrowband, regulated and unforgiving, so learn the plumbing. Connect through native integrations, SIP trunks or media streaming. Design inbound flows with disclosure, CRM lookups and warm transfers. Treat outbound with extra care: consent, time zones, voicemail and suppression. Your next step: connect a test number to your agent, call it from a real mobile in a noisy room, and try a transfer. Then look at the lesson text for the outbound call code, including the consent check.
Why telephony is its own skill
A web voice widget talks over the internet with good audio. A phone call crosses the public switched telephone network (PSTN): narrowband audio (often 8 kHz), codecs, carrier routing, caller ID rules, regional regulations, and the expectation that it just works. Most business voice agents live on the phone, so understanding the plumbing saves you days of debugging.
Key concepts
| Term | Meaning |
|---|---|
| DID / phone number | A number people dial; bought from a carrier or CPaaS provider (for example Twilio, Vonage, Telnyx, Plivo) |
| CPaaS | Communications Platform as a Service: APIs for numbers, calls and messages |
| SIP | Session Initiation Protocol: the signaling standard for internet telephony (setting up, transferring and ending calls) |
| SIP trunk | A connection between your phone system or carrier and another SIP endpoint (for example your voice agent platform) |
| RTP / media | The actual audio stream, separate from signaling |
| Codec | Audio encoding such as G.711 (mu-law/A-law, 8 kHz), common on phone networks |
| DTMF | Keypad tones ("press 1") |
| Warm vs cold transfer | Warm: agent or system briefs the human first (or conferences in). Cold/blind: call passed directly |
| SIP REFER | A SIP method to transfer a call to another destination |
Three ways to connect an agent to phones
- Native platform integration. Platforms like ElevenAgents let you import a Twilio number (or buy/assign numbers) and route inbound calls to an agent, plus place outbound calls via API. Fastest path.
- SIP trunking. Point an existing phone system, contact center platform or carrier trunk at the agent platform's SIP endpoint. Best when you already have numbers and a PBX or contact center (for example keeping your UAE or Pakistan numbers with a local carrier). OpenAI's Realtime API and LiveKit also accept SIP.
- Media streaming via CPaaS. Twilio's Media Streams send raw call audio over WebSocket to your server, and Twilio's ConversationRelay handles STT and TTS for you and sends text over WebSocket to your LLM app. You write more code but control everything.
Inbound call flow (typical)
Caller dials +971 4 XXX XXXX
-> Carrier -> CPaaS number (e.g. Twilio)
-> Webhook or SIP routing -> Voice agent platform (agent_id)
-> Agent greets with AI disclosure (and recording notice if recording)
-> Tools (CRM lookup by caller ID, booking API)
-> Resolve, or transfer to human (SIP REFER / conference) with context
-> End call -> post-call webhook with transcript, evaluation, data fieldsOutbound calls
Outbound use cases include appointment reminders, lead follow-up and payment reminders. Outbound carries the heaviest legal risk (consent, calling hours, do-not-call lists, AI disclosure, caller ID rules), covered in Module 5. Technically you:
- Load the contact and consent status from your CRM.
- Pass dynamic variables (name, appointment time, reason) at call start.
- Enable voicemail detection to leave a short message or hang up.
- Handle retries politely (limits per day, respect time zones).
- Log outcomes back to the CRM.
Many platforms also support batch calling for campaigns; add pacing and suppression lists.
Transfers that do not frustrate people
- Tell the caller what is happening ("I'm connecting you to Sara at reception now").
- Pass context: summary, caller details, reason. SIP headers or user-to-user information can carry IDs; many platforms send a summary to the human's screen or CRM.
- Have a fallback when nobody answers: take a callback number, or offer a booking.
- Respect business hours: outside hours, do not transfer to an empty line.
Audio quality on phone lines
- Phone audio is narrowband: TTS voices that sound great on headphones can sound thin. Test on real phones.
- Background noise: noise suppression helps ASR; ask callers to repeat politely, not repeatedly.
- DTMF: let callers type account numbers on the keypad where speech recognition struggles (and the agent can send tones to navigate other IVRs).
Worked example: a Riyadh car-service center
Inbound: a Saudi number with a local carrier connects via SIP trunk to the voice agent platform. The agent answers in Arabic by default, detects English speakers, books service slots through the dealership management system API, and transfers to the service desk via SIP REFER during working hours with a summary. Outbound: day-before reminders to customers who opted in on their service contract; voicemail detection leaves a short Arabic message with a callback number. All calls carry the AI disclosure in the first sentence; data stays in regions compliant with KSA PDPL transfer rules (verify vendor processing locations).
Hands-on: place an outbound call through ElevenAgents with Twilio
import os
from elevenlabs.client import ElevenLabs
client = ElevenLabs(api_key=os.environ["ELEVENLABS_API_KEY"])
def remind(to_number: str, name: str, appt_time: str, consent_ok: bool):
if not consent_ok:
raise ValueError("No recorded consent for automated calls to this number")
resp = client.conversational_ai.twilio.outbound_call(
agent_id=os.environ["REMINDER_AGENT_ID"],
agent_phone_number_id=os.environ["AGENT_PHONE_NUMBER_ID"], # the imported Twilio number in ElevenAgents
to_number=to_number, # E.164 format, e.g. +9715XXXXXXXX
conversation_initiation_client_data={
"dynamic_variables": {"customer_name": name, "appointment_time": appt_time}
},
)
return respCheck the current SDK reference for parameter names; there is an equivalent SIP trunk outbound method. Wrap calls with consent checks, calling-hour checks for the recipient's time zone, and suppression-list lookups.
Pitfalls
- Testing only on laptop speakers; phone audio behaves differently.
- Cold transfers with no context, so callers repeat everything.
- Outbound campaigns without consent records, calling-hours logic and suppression lists.
Key takeaways
- Phone calls use narrowband audio over the PSTN; test voices, ASR and turn-taking on real phones.
- Connect agents via native CPaaS integrations (e.g. Twilio import), SIP trunks from existing carriers or PBXs, or media streaming (Twilio Media Streams/ConversationRelay).
- Inbound flows need disclosure, CRM lookup and warm transfers with context and fallbacks.
- Outbound calls need consent checks, dynamic variables, voicemail detection, time-zone-aware retries and suppression lists.
Check your understanding
Quick questions to lock in the lesson. They don’t count towards your certificate.
Put it into practice
Connect a test number to your agent. Call it from a real mobile in a noisy room, try a transfer, and note any ASR or turn-taking issues. Implement the outbound call function with consent and calling-hours guards.
Enrol for free to save your progress
Reading is always free. Enrol to keep your place, take the final assessment and earn a verifiable certificate.