Contact center operations teams often spend significant time tuning NLU routing prompts, then discover containment is still low because the underlying speech-to-text engine returns garbled transcripts on accented speech or runs 1.5 seconds behind the conversation. The NLU layer gets the blame, but the STT layer is where the problem starts.
Legacy DTMF (dual-tone multi-frequency) menus compound this by forcing callers into rigid branches that do not match how they describe their problems. Any mismatch ends in hang-ups, misroutes, or repeat calls. Conversational AI IVR fixes the structural problem by routing on intent rather than key presses, but only when the audio infrastructure underneath is fast enough and accurate enough to capture what the caller actually said. This article breaks down how real-time speech-to-text transforms touch-tone menus into dynamic, intent-driven routing systems and outlines the technical thresholds required to deploy them against your existing telephony stack.
Moving beyond DTMF to intent-driven routing
DTMF routing is deterministic by design: the caller presses a key, the system branches to a predetermined menu node. That predictability is also its ceiling. The caller's problem must map exactly onto a branch someone anticipated when the tree was built, and when it does not, the system loops or defaults to an agent queue.
Conversational AI IVR replaces that binary model with a three-layer architecture.
Core elements of conversational IVR
- Real-time STT (the ears): Audio streaming from the SIP trunk is forked at the RTP layer and passed to the transcription engine, which returns a text stream in near real-time while the original call continues without interruption.
- NLU or LLM intent parser (the brain): The intent parser analyzes the text stream to identify the caller's primary intent and any entities present, such as account numbers, dates, or product names, before routing logic executes.
- Dynamic routing webhook (the hands): Based on the extracted intent and entities, the system triggers a routing decision, a self-service action, or a warm handoff with full context to the live agent's screen.
The first layer fails most often in production. Operations teams spend months tuning NLU routing prompts, only to discover containment stays low because the STT engine returns garbled transcripts on accented speech or runs 1.5 seconds behind the conversation.
AI IVR vs. traditional routing
| Dimension |
Standard DTMF IVR |
Conversational AI IVR |
| Input method |
Touch-tone key presses |
Natural language speech |
| User experience |
Rigid menu navigation, multiple hops |
Single utterance, intent extracted immediately |
| Routing logic |
Predetermined branches, fixed options |
NLU entity extraction, dynamic decision trees |
| Flexibility |
High change effort, requires menu rebuild |
Intent model updates with reduced infrastructure changes |
DTMF still has a role for simple binary inputs, such as a caller confirming their language preference or entering a PIN. The safest deployment combines voice input with touch-tone fallback for accessibility and noisy environments. Conversational IVR augments DTMF rather than replacing it entirely.
Upgrading DTMF to conversational AI IVR
The hidden costs of DTMF routing
Legacy menu trees inflate cost-per-contact in ways that do not appear on a single line item. Callers who select the wrong menu option get transferred, and each transfer adds agent handle time and increases the probability of a repeat call. Menu navigation time before a caller reaches an intent-matched queue adds to queue occupancy without adding to resolved volume. Abandonment during long menu sequences means the original inbound cost is absorbed with zero resolution.
The financial profile compounds for multilingual queues. A contact center routing Spanish, Tagalog, or Tamil speakers through an English-language DTMF tree with a single "press 2 for Spanish" branch creates friction before the conversation even starts.
How intent routing improves call flow
When a caller states their problem in plain language, the conversational IVR extracts intent in a single turn rather than requiring three or four menu hops. Key data extraction from calls, such as account numbers, policy IDs, or transaction dates, can happen within that same first utterance, eliminating the discovery phase for the agent. Prodigal Tech's containment benchmarks put beginner AI IVR deployments at 20–40% containment and intermediate/mature deployments at 40–70%, though figures vary considerably by vertical, intent complexity, and STT accuracy. Both figures outperform DTMF systems on complex multi-language queues where menu mismatches are structurally unavoidable.
Scaling IVR deflection with AI
High agent attrition, which industry benchmarks put at 55–60% annually in high-stress contact center environments, strains capacity continuously. Self-service containment is the most direct pressure valve: every call resolved by the IVR without agent involvement reduces both the volume hitting queues and the per-agent productivity requirement during peak periods. High-volume, low-complexity intents, such as order tracking, balance inquiries, password resets, and appointment confirmations, are the clearest containment candidates for CCaaS platforms.
How speech-to-text enables conversational IVR
Replacing DTMF with conversational input
At the integration layer, a Session Border Controller forks the audio stream and passes a copy to the real-time STT engine without interrupting the live call. The STT engine returns a text stream with word-level timestamps and confidence scores that the NLU layer uses for routing decisions.
Partial transcripts, which are intermediate text outputs returned before an utterance is complete, allow the NLU engine to begin intent scoring before the caller finishes speaking. This overlap is what makes conversational latency feel natural rather than mechanical.
Latency thresholds for natural IVR flow
In natural human conversation, the average gap between one speaker finishing and another beginning runs to approximately 200 milliseconds, a threshold well-documented in turn-taking literature. In production IVR environments, gaps beyond 300 to 400ms cause callers to perceive the system as unresponsive. Beyond 500ms, they assume the system did not hear them and begin repeating themselves, which triggers intent-recognition loops and destroys CSAT scores.
For conversational IVR to avoid those loops, the real-time STT engine needs to return partial transcripts fast enough for the NLU layer to begin processing before the utterance ends. We built Solaria-1 to deliver partial transcriptions under 103ms, with a final transcript latency of approximately 300ms, keeping the perceived response within the range where the system feels conversational.
Balancing intent accuracy and CX
Speed without accuracy destroys containment just as reliably as high latency. If the STT engine returns fast but garbled text on accented speech, the NLU engine misroutes the call and the caller either loops, gets transferred to the wrong queue, or abandons. Entity-level accuracy matters more than aggregate word error rate (WER) for IVR routing: whether critical tokens like account numbers, names, and stated intents survive transcription correctly determines whether the routing decision is right, not just whether the overall word count is approximately correct.
Practical conditions in contact center environments, including cell phone distortion, background noise, and overlapping speech, consistently degrade models trained primarily on clean studio recordings. Production testing shows accuracy varies substantially across acoustic conditions that all exist within a single contact center, as clean headset calls, conference-room echo, and noisy mobile calls each degrade differently across the STT providers you are evaluating.
Optimizing call flow using conversational AI
Improving routing via AI transcripts
Structured, real-time transcripts give the routing layer more than an intent label. They pass extracted entities, such as account numbers, product names, dates, and stated urgency, directly into the routing decision. A caller who says "I need to dispute a charge from last Thursday on my Visa ending in 4421" has already provided the routing target, the account reference, and the transaction scope before the first menu hop would have completed in a DTMF system.
Entity extraction before routing also reduces the discovery phase for agents who receive escalated calls, cutting the time agents spend asking for information the IVR already captured. Structured transcript outputs pass to downstream systems, including CRM webhooks and routing APIs, via our audio-to-LLM pipeline.
How NLU replaces rigid IVR menus
The most reliable production deployments combine LLM-based intent understanding with deterministic fallback logic. A pure LLM approach introduces hallucination risk: the system may infer an intent the caller did not express and route to a destination that creates further frustration. A hybrid approach uses the LLM layer for natural language understanding while deterministic rules govern the final routing action and confirmation steps.
Recovery patterns matter as much as the primary routing path. When an intent falls below the confidence threshold, the system should offer a clarifying prompt rather than defaulting immediately to an agent queue. One clarification loop costs a few seconds. An unnecessary agent transfer costs AHT and often triggers a second call.
Solving multi-intent routing failures
Callers frequently state multiple problems in a single utterance: "I need to pay my bill and update my address." Single-intent parsers break on these inputs and typically route to whichever intent scores highest, leaving the second issue unaddressed and driving a follow-up call. Multi-intent parsing extracts each discrete intent and sequences them, either handling both within the IVR, routing to the correct queue for the primary intent while logging the secondary, or flagging both for the agent screen during handoff. This requires the STT layer to produce complete, accurate transcripts because partial parsing of a garbled transcript compounds routing errors.
Retain more callers with real-time AI language support
Reducing drop-off in non-English queues
Legacy IVR systems handle language routing through a single upfront prompt: "Press 2 for Spanish." Systems designed for monolingual input cannot handle callers who code-switch mid-sentence, mixing languages within a single utterance. The operational consequence is measurable drop-off: callers who encounter a system that cannot understand their language abandon the queue and call back with lower CSAT, or escalate through more expensive channels like chat or email.
Multilingual BPO operations in Southeast Asia, South Asia, and Latin America face this at scale. Agents handle calls in regional languages and dialects, and any QA or analytics layer built on transcripts produced by an English-centric STT model will be unreliable for non-English interactions.
Automating language routing in AI IVR
Solaria-1 detects the caller's language automatically from their speech in the first few words using automatic language detection, routing the transcript accordingly without requiring the caller to interact with a language selection prompt. A Spanish speaker who calls and simply begins speaking is detected and routed to the Spanish-language queue without any menu friction.
This detection operates across 100+ supported languages, removing the "press 2 for Spanish" ceiling and extending automatic routing to languages that most DTMF systems never cover. For BPO operations handling Tagalog, Bengali, Tamil, Urdu, or Persian speakers, that language coverage gap shows up directly in containment data and QA scoring reliability.
Solving dialect challenges in speech AI
Accented speech consistently degrades models trained primarily on American English audio, translating directly into misrouted calls, corrupted entity extraction, and QA data that cannot be trusted for coaching or compliance.
We built Solaria-1 to handle accented and multilingual speech in production, including code-switching where callers alternate between languages mid-conversation without the transcript breaking or the session degrading.
"Gladia provides a highly accurate real-time speech-to-text solution for high volumes of support and service calls. Latency is low and accuracy high, even for numericals." - Verified user review on G2
Scaling AI IVR without breaking existing workflows
Mapping AI IVR to current call routing
The most common integration pattern taps the RTP media layer using a Session Border Controller or media proxy. The SBC creates a parallel copy of the RTP stream and passes it to the STT API while leaving PBX call control untouched. This means the AI IVR layer sits in front of your existing SIP/PSTN infrastructure, including CCaaS (Contact Center as a Service) platforms like Genesys, Twilio, Avaya, or Vonage, without requiring a telephony migration.
Validating AI IVR with production audio
Before committing to a real-time STT vendor for a regulated CCaaS deployment, run an operational readiness evaluation against your actual call audio, not a vendor-supplied demo dataset.
Operational readiness checklist:
- SIP and RTP compatibility: Confirm the STT API accepts your specific codec encoding (G.711 µ-law, G.711 A-law, or Opus) and integrates at the RTP layer without call path modification.
- Latency budget and QA output: Measure P95 final transcript latency on your call audio under load, targeting P95 final latency at or below 300ms. Verify that structured transcript outputs include word-level timestamps and confidence scores sufficient for your automated QA scoring layer.
- Multilingual and accent performance: Test with a representative sample of your actual caller audio, including accented speech, background noise, and code-switching, not clean benchmark recordings.
- Data residency and compliance: Confirm that your audio is processed in a geographic region consistent with your compliance requirements, with a signed Data Processing Agreement available before you send any live call audio. Check the vendor's uptime and SLA history before committing to a production deployment.
Strategies for context-rich agent handoffs
When intent recognition determines a call requires escalation, the real-time transcript and extracted entities must transfer with the caller to the agent's screen before the first word is spoken. This prevents the caller from repeating their account number, the nature of their problem, or the actions already attempted in the IVR.
Implementing this requires the routing webhook to pass a structured payload, including the full transcript, the identified intent, the extracted entities, and the IVR actions taken, to the agent desktop or CRM via API before the call connects.
Data governance for AI voice tools
Voice recordings qualify as personal data under GDPR Article 4. Processing them through an uncertified STT API creates compliance exposure that surfaces during audits rather than procurement. Every certification gap in your STT vendor's stack becomes your liability.
| Certification |
Requirement |
Our status |
| GDPR |
Signed DPA, EU-hosted processing, no model training on customer audio by default on Growth and Enterprise plans |
Active. Growth and Enterprise: no training on customer audio, no opt-out required. |
| SOC 2 Type II |
Active report covering the production API and data handling |
Active report. |
| ISO 27001 |
Active certificate with current surveillance audit date |
Active certificate. |
| HIPAA |
BAA available before any PHI is processed |
BAA available. |
| PCI DSS |
Required for any call flow handling payment card data |
Certified. |
Quantifying AI impact on call center efficiency
Boosting AI IVR containment rates
Containment rate measures the percentage of inbound calls resolved by the IVR without live-agent involvement. Accurate real-time STT directly raises this metric because the NLU engine can only route correctly on intents it receives correctly. A garbled transcript on a containable intent, such as a balance inquiry or appointment confirmation, forces an unnecessary transfer and lowers containment.
Quantifying AHT gains from AI IVR
Pre-identification of caller intent and entity extraction before agent handoff cuts AHT by eliminating the discovery phase. Even partial containment, where the IVR collects intent and account data before routing to a human, removes 30 to 60 seconds from the average interaction by cutting the agent's opening verification and problem-discovery sequence. The gains vary considerably by vertical, intent complexity, and what share of call volume is fully contained versus partially handled, which is why modeling AHT impact against your own call mix produces more reliable projections than industry averages.
Automating inbound intent resolution
When the STT and NLU layers resolve a call entirely within the IVR, the cost-per-contact drops from live-agent costs, which MaestroQA's analysis of 18 large enterprise call centers (call volumes 900K–9M) puts at $2.70 to $5.60 per call, to a small fraction of that per voice AI-contained call, since the marginal cost of an automated resolution is close to pure compute cost rather than agent time. Figures vary by deployment size, call complexity, and vertical. At scale, that spread is the largest single lever for reducing total contact center operating cost without proportional headcount growth.
Addressing common AI IVR implementation risks
Defining accuracy for AI IVR success
WER figures from clean-audio benchmarks do not predict production performance in contact center environments. This is why our async benchmark tests Solaria-1 against 8 providers across 7 datasets and 74+ hours of audio rather than clean read-speech alone. Across those async testing conditions, Solaria-1 delivers on average 29% lower WER than alternatives on conversational speech. The metric that matters for IVR routing is entity-level accuracy: whether critical tokens like account numbers, names, product identifiers, and stated intents survive transcription correctly. Testing on your own historical call recordings before committing to a vendor is the only reliable method.
Improving STT accuracy in noisy calls
We engineered Solaria-1 to reduce hallucinations on noisy audio, a failure mode where transcription models insert words or phrases that were never spoken when audio quality degrades. Background noise, cell phone distortion, and overlapping speech are the primary triggers in production IVR environments. Applying a noise gate or acoustic pre-processing filter to the RTP fork before it reaches the transcription engine improves accuracy on the worst-quality inbound audio without changing the main call path.
Integrating AI IVR with legacy telephony
Choose your integration approach based on scope and existing technical debt:
- Rapid deployment (no-code wrappers): Plug-and-play SIP connector products sit in front of an existing IVR platform and route the audio stream to the STT API. Integration timeline is typically four to eight weeks and suits operations testing conversational IVR on a subset of queues before full rollout.
- Enterprise orchestration (custom SIP/RTP integration): Full media-layer integration with custom NLU pipeline, entity extraction, and CRM webhook architecture. Integration timeline ranges from six to 18 months depending on legacy system complexity. If you're migrating from another STT provider, our migration guide from Deepgram and migration guide from AssemblyAI cover the specific API changes required.
Calculating your true cost per contact
When intent recognition confidence falls below threshold, the system should attempt one clarification prompt before routing to a human agent. One structured recovery prompt is the standard production pattern. Routing to a human on the first recognition failure wastes containment capacity and inflates AHT.
Understanding your true cost-per-contact for AI IVR requires accounting for each layer of the stack, not just the base STT rate.
| Cost line |
Range |
Notes |
| Real-time STT |
$0.25 to $0.75/hr |
Starter: $0.75/hr. Growth: as low as $0.25/hr. Enterprise: custom pricing. |
| Diarization |
$0 on Starter and Growth |
Applies to async post-call analysis, not the live IVR flow. Some providers charge separately; we bundle this at the base rate. |
| Translation |
$0 on Starter and Growth |
Charged as an add-on by some competitors. |
| Sentiment analysis |
$0 on Starter and Growth |
Deepgram and AssemblyAI price this separately per their public pricing. |
| Agent handoff cost (failed containment) |
$2.70 to $5.60 per call |
Per MaestroQA's analysis of 18 large enterprise call centers (call volumes 900K–9M): $2.70–$5.60 per call. Complex and escalated calls typically cost more; no verified public benchmark found for that tier. Figures vary by deployment size and vertical. |
| AI IVR resolved cost |
A small fraction of live-agent cost |
Marginal cost of an automated resolution is close to pure compute cost rather than agent time. No verified third-party figure available; model against your own infrastructure cost. |
Providers that advertise competitive base STT rates often add diarization, translation, and sentiment as separate billable features; Deepgram and AssemblyAI both price sentiment separately per their public pricing, producing a materially higher effective cost at scale. We include all audio intelligence features in the base rate on our Starter and Growth plans.
Get started with Gladia and have your integration in production in less than a day, or test Solaria-1 on your own multilingual audio to see how it handles language detection, accent-heavy speech, and code-switching directly against your production call conditions.
FAQs
What is the maximum latency allowed for conversational IVR?
To maintain a natural conversational flow, the total round-trip latency (STT plus NLU plus response) should remain under 500 to 800ms, which requires the real-time STT engine to deliver partial transcripts under 300ms so the NLU layer can begin processing before the utterance ends. Solaria-1 delivers partials under 103ms and a final transcript latency of approximately 300ms.
Does conversational IVR support touch-tone fallback?
Yes, our system supports DTMF input alongside voice, so callers in noisy environments or those who prefer keypad input can still interact without being forced into voice-only flows. Hybrid input support is also the recommended approach for accessibility compliance.
Is caller data used to train your speech models?
On our Growth and Enterprise plans, customer audio and transcripts are never used for model training, and no opt-out action is required. On our Starter plan, data can be used for training by default.
How does code-switching affect IVR routing accuracy?
Code-switching causes most monolingual STT engines to misidentify the language or produce garbled output for non-primary language segments. Solaria-1 handles mid-conversation language changes natively, maintaining transcript integrity so the NLU layer receives usable intent data regardless of where the language shift occurred.
What compliance certifications are required for voice data in regulated industries?
The mandatory baseline for regulated contact centers is GDPR (including a signed Data Processing Agreement with EU-hosted processing options), SOC 2 Type II (active report covering the production API), ISO 27001 (active certificate), and HIPAA (BAA in place before any PHI is processed). PCI DSS certification is required if any call flow captures payment card data. We hold all five.
Key terms glossary
CCaaS (Contact Center as a Service): A cloud-based customer service platform that provides call center functionality without on-premises hardware, enabling companies to manage customer interactions across multiple channels.
PBX (Private Branch Exchange): A private telephone network used within an organization that allows users to communicate internally and externally while managing call routing, voicemail, and other telephony features.
PSTN (Public Switched Telephone Network): The traditional circuit-switched telephone network that connects phones worldwide, operating through physical telephone lines and switching centers.
RTP (Real-time Transport Protocol): The network protocol used to deliver audio and video streams over IP networks, commonly used in VoIP systems to carry the actual voice data during phone calls.
P95 latency: The 95th percentile latency measurement, indicating that 95% of requests complete faster than this threshold. Used to measure performance while excluding extreme outliers that may skew average metrics.
Word Error Rate (WER): The standard metric for speech recognition accuracy, calculated by dividing the total number of insertions, deletions, and substitutions in the transcript by the total number of words spoken. Lower WER indicates higher transcription accuracy.
Diarization Error Rate (DER): The metric used to evaluate speaker attribution accuracy in async (post-call) transcription workflows, measuring the percentage of audio time where a transcript segment is assigned to the wrong speaker. It is not applicable to the live IVR call flow. Lower DER in post-call analysis means transcripts more reliably attribute each segment to the correct speaker, which matters for QA scoring, compliance review, and coaching workflows.
Code-switching: The practice of alternating between two or more languages or dialects within a single conversation or sentence, common in multilingual contact center environments and BPO operations serving diverse caller populations.
DTMF (dual-tone multi-frequency): The touch-tone signaling system used to send key-press input over telephone networks, historically the primary input mechanism for IVR menu navigation.
NLU (natural language understanding): The component of a conversational AI system that interprets the meaning and intent of a spoken or written input, operating on the text stream produced by the STT engine to determine routing decisions.
Containment rate: The percentage of inbound calls fully resolved by an IVR or self-service system without requiring transfer to a live agent, the primary measure of IVR automation effectiveness in contact center operations.
