Before choosing a real-time transcription API, most product teams should ask a different question: do you actually need real-time? For meeting assistants, post-call analytics, and note-taking tools, async transcription delivers higher accuracy, better speaker attribution, and full multilingual context because the model processes the complete recording before generating output. Real-time transcription is the right architectural choice for a specific set of use cases: voice agents, live captions, and live agent assist tools where sub-300ms latency is a functional requirement, not a nice-to-have.
If your product falls into that second category, this guide covers how to evaluate real-time STT infrastructure, balance latency with production-grade Word Error Rate (WER), and deploy audio infrastructure that won't break under production conditions. In production environments, optimizing for latency alone often introduces accuracy trade-offs that create more downstream failures than a slightly slower but more consistent pipeline.
Real-time vs. async: choosing the right architecture
Do you need real-time or async transcription?
The decision comes down to one functional question: does your product need output during the conversation, or after it? If a user is waiting for a response mid-sentence (as in a voice agent or live caption feed), sub-300ms latency is a hard requirement and real-time is the only viable path. If the output is consumed after the conversation ends (meeting notes, post-call QA scores, CRM entries), async transcription gives the model full audio context, which improves accuracy, diarization, and multilingual consistency. Getting this wrong means rebuilding the audio pipeline later, not just swapping a config value.
| Use case |
Right architecture |
Why |
| Meeting notes, post-call analytics, CCaaS QA |
Async (batch) |
Full audio context produces higher accuracy, better diarization, and fewer hallucinations |
| Voice agents, conversational AI |
Real-time (streaming) |
Sub-300ms latency required for natural turn-taking |
| Live captions, accessibility |
Real-time (streaming) |
Immediate output required for viewers |
| Live agent assist |
Real-time (streaming) |
On-screen prompts must appear during the call |
If your use case is in the first row, start with our async meeting assistant architecture guide instead. The rest of this guide focuses on rows 2-4.
Immediate vs. delayed transcription
Real-time streaming and asynchronous (batch) processing serve different jobs and come with different accuracy trade-offs.
| Mode |
Latency |
Best for |
| Real-time (streaming) |
~270ms (Solaria-1) |
Voice agents, live captions, live agent assist |
| Async (batch) |
Seconds to minutes |
Meeting notes, post-call analytics, CCaaS QA |
| Human transcription |
4-6 hours per audio hour |
High-stakes legal or medical review |
Async systems typically have access to the full audio context before generating output, which can improve punctuation accuracy, word disambiguation, and diarization quality. Real-time systems generate output within a few hundred milliseconds with limited context. That trade-off is acceptable when immediate output is a functional requirement, as in voice agents and live captions.
The real-time STT pipeline
A production real-time pipeline streams audio over WebSocket to the inference endpoint, where the model generates two output types: partials (provisional transcriptions optimized for speed) and finals (definitive transcriptions after a clear speech endpoint). Solaria-1 delivers an average time to first final of 270ms for real-time streams.
Real-time call and meeting transcription
Real-time STT delivers the most value in three production contexts:
- Live agent assist: Contact center agents can receive real-time support during live calls.
- Voice agents: Conversational AI systems typically need transcription with low latency to maintain natural turn-taking.
- Live accessibility captions: Real-time captions for meetings, webinars, and broadcast media provide continuous output for viewers with hearing impairments.
Scaling production: Cost and reliability
Getting real-time transcription to work in a demo is straightforward. Getting it to hold up at 1M calls per week, across multiple languages, with stable WER and predictable infrastructure costs is where most teams discover gaps in their original vendor choice.
Achieving sub-second transcription speed
A latency budget is the total time from when a user finishes speaking to when a downstream system receives actionable output. Transcription latency is one component of that budget, alongside other factors such as network round-trip time and downstream processing steps.
For voice agents, the practical threshold for perceived naturalness is a total response latency below 500ms, which means the STT layer needs to deliver finals well under 300ms to leave room for LLM inference and TTS output. Solaria-1's ~270ms average response time is designed to support this requirement.
Preventing live WER regressions
WER in a real-time stream is not a fixed number. It degrades under conditions common in production environments: background noise from open offices or call centers, and compressed telephony audio at 8kHz (standard for PSTN).
Preventing cost overruns at scale
The most common source of infrastructure cost surprises in real-time STT is not the base rate. It's the feature add-ons that aren't visible until the first invoice at scale. Providers with unbundled pricing can see costs increase significantly once features like speaker identification, sentiment analysis, entity detection, and summarization are each priced separately.
Our Starter and Growth plans bundle audio intelligence features including sentiment analysis, summarization, and translation into the base rate. That pricing model makes cost projection at 10x volume a single multiplication, not a spreadsheet exercise with footnotes.
Navigating real-time transcription challenges
Every engineering team building on real-time STT encounters the same production challenges. The difference between teams that ship reliably and teams that spend months debugging accuracy regressions is knowing which challenges to mitigate at the infrastructure layer.
Noisy audio's impact on WER
Background noise is a common cause of WER degradation in production real-time environments. Call center floors, remote workers on laptop microphones, and mobile callers in transit all introduce noise profiles that can push error rates into double digits on models not trained for adversarial conditions.
Mitigation involves two layers: model-level robustness (models trained on diverse, noisy datasets) and optional audio preprocessing (noise suppression applied before audio reaches the inference endpoint). Solaria-1 is designed for noisy call center environments, reducing the need for upstream audio cleaning in most production workflows.
Transcribing code-switching speech
Code-switching, where a speaker moves between languages within a single utterance, is a normal communication pattern in multilingual call centers, global SaaS products, and mixed-language team meetings. It's also one of the most reliable ways to break a legacy ASR model. Our guide on code-switching in speech recognition explains why it creates systematic failures in English-first models.
The structural problem is training data bias. Models trained primarily on English typically have less capacity for non-English patterns, and minimal capacity for the transitions between languages that define code-switching. When a caller switches from English to Tagalog mid-sentence, a monolingual model either drops the non-English segment entirely or produces garbled output, with no visible error signal to the product or the user.
Solaria-1 handles code-switching natively across all 100+ supported languages. The model continuously detects the spoken language and switches transcription accordingly, without requiring callers to announce the change. The code-switching contact center guide details exactly where in the pipeline these failures occur and how to prevent them.
Speaker attribution for real-time pipelines
Speaker diarization, which identifies who said what across a multi-speaker recording, is more challenging in real-time environments because the model must make speaker attribution decisions with partial context.
Speaker attribution is best handled in post-processing, in an async workflow where the complete recording is available.
Overcoming transcription hallucination errors
In a voice agent context, a hallucinated phrase can trigger an incorrect intent classification. In a live agent assist tool, it can display wrong information on the agent's screen during an active customer call.
Modern architectures mitigate this with techniques like VAD gating (inference runs only when speech is detected), confidence thresholding (low-confidence outputs are filtered before display), and custom vocabulary support to reduce substitution errors on domain-specific terms. Our real-time engine supports custom vocabulary, so product-specific terms, brand names, and technical jargon are recognized correctly rather than replaced with acoustically similar common words.
Gladia's AI for production-grade transcripts
How we benchmark realistic WER
Our benchmark methodology is open and reproducible at our async STT benchmark page. We evaluate Solaria-1 against providers including AssemblyAI, Deepgram, AWS Transcribe, Azure Speech, Google Speech-to-Text, Rev AI, and Speechmatics across 7 datasets and more than 74 hours of audio, covering diverse accents, dialects, noisy conditions, and conversational speech. We specifically avoid testing only on a single version of Common Voice, which is a common way for providers to optimize benchmark numbers rather than production performance.
Solaria-1 achieves on average 29% lower WER than alternatives on conversational speech and on average 3x lower DER (diarization error rate) in async workflows. In production environments, Claap (a video meeting platform) reached 1-3% WER using Gladia, with one hour of video transcribed in under 60 seconds on the async pipeline.
Production-grade accuracy for 100+ languages
Solaria-1 supports 100+ languages and dialects, including languages that have limited coverage through other API-level STT providers. That coverage includes Tagalog, Bengali, Punjabi, Tamil, Urdu, Persian, Marathi, Haitian Creole, Maori, and Javanese, which are languages that matter for BPO operations in Southeast Asia, South Asia, and Latin America. As described in our Solaria-1 launch post, we built this model to treat multilingual robustness as a primary constraint, not an edge case.
Meeting your sub-300ms latency budget
Solaria-1 delivers an average response time of 270ms for real-time streams. That latency profile fits the requirements for voice agents (sub-300ms to maintain conversational turn-taking) and live-assist tools (sub-second display for agent reference during calls).
To put this in production context, Aircall processes over 1M calls per week through Gladia, cutting transcription processing time by 95%. That's not a benchmark result; it's sustained infrastructure performance at scale.
Speaker attribution in real-time workflows
Gladia's speaker diarization runs on pyannoteAI's Precision-2 model and is available in async mode only. The model uses the complete recording to improve speaker attribution accuracy, as documented in our speaker diarization docs. For the best speaker attribution results, we recommend using the async diarization pipeline where the model has access to the complete speaker profiles before making labeling decisions.
Integration time and engineering velocity
Sub-24-hour integration window
Our real-time API uses WebSocket, with lightweight Python and JavaScript SDKs that cover session initialization, audio streaming, and result handling. Native integrations with voice agent frameworks (LiveKit, Pipecat, Vapi) let teams pass audio directly without building a custom routing layer. The getting started documentation covers the full initialization flow, and the real-time React app tutorial shows a complete TypeScript implementation.
If you're migrating from a different provider, we maintain dedicated migration guides for moving from Deepgram and moving from AssemblyAI, covering the API differences and configuration mappings that typically take the most time during a switch.
Fast-track API integration code
Here's a typical WebSocket session initialization pattern:
// Initialize a real-time WebSocket session
const response = await fetch('https://api.gladia.io/v2/live', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-Gladia-Key': '<YOUR_GLADIA_API_KEY>',
},
body: JSON.stringify({
encoding: 'wav/pcm',
sample_rate: 16000,
bit_depth: 16,
channels: 1,
}),
});
const { url } = await response.json();
const socket = new WebSocket(url);
// Stream audio over the WebSocket connection
socket.addEventListener("open", function() {
socket.send(audioChunk);
});
The integration follows a standard pattern: initialize the session, stream audio over WebSocket, and receive transcription results.
Peer case studies for faster integration
The integration timeline is production-verified. Scoreplay reported less than a day from dev work to production release. Attention runs Gladia as its core transcription layer for CRM population and coaching scorecards. As their team put it: "Reactive support helps us ship faster."
Sustainable transcription costs at scale
Precise per-hour usage billing
We bill based on audio duration processed.
Current published rates:
| Plan |
Real-time |
Async |
Features included |
| Starter |
$0.75/hr |
$0.61/hr |
All audio intelligence features |
| Growth |
From $0.25/hr |
From $0.20/hr |
All audio intelligence features; rate decreases with upfront volume commitment |
| Enterprise |
Custom |
Custom |
Custom models, fine-tuning, debundled pricing |
Avoid unexpected feature fees
Our base rate on Starter and Growth includes features like translation across 100+ languages, sentiment analysis, and summarization. When you model costs at 10x your current volume, there are no additional line items to discover.
For context, Deepgram and AssemblyAI price features like sentiment analysis, entity detection, summarization, and speaker identification as separate add-ons per their published pricing, bringing the effective rate higher than the advertised base rate. The gap between a bundled rate and a base-plus-add-ons structure becomes a significant line item when you multiply it across 10,000 hours per month.
For teams building on voice agent frameworks, the end-to-end voice agents webinar covers how combining STT with downstream TTS and LLM layers affects total pipeline cost and where the Gladia layer fits within that budget.
Start with 10 free hours and have your integration in production in less than a day. Run Solaria-1 against your own production audio and compare the WER on accented speech, code-switching, and noisy call conditions against any other provider you're evaluating.
FAQs
How does Gladia benchmark WER on noisy call audio?
For async workflows, we benchmark Solaria-1 against 8 providers across 7 public datasets covering more than 74 hours of audio, including diverse accents, dialects, and noisy conditions. On conversational speech, Solaria-1 achieves on average 29% lower WER than competing APIs. For real-time performance benchmarks, see our real-time STT benchmark.
What is Gladia's audio retraining policy?
On Growth and Enterprise plans, customer audio is never used to train models, no opt-out required. On Starter, data may be used for training by default.
What are Gladia's data residency policies?
We offer deployment options with regional data residency. Certifications include SOC 2 Type II and ISO 27001, with GDPR compliance and HIPAA alignment for regulated industries.
How does Gladia integrate with voice agent APIs?
We natively integrate with Pipecat, LiveKit, and Vapi, allowing teams to connect audio from these frameworks to the Solaria-1 real-time pipeline.
Key terms glossary
Word Error Rate (WER): The percentage of words in a transcript that differ from the reference text, calculated as substitutions plus deletions plus insertions divided by total reference words. Lower is better.
Diarization Error Rate (DER): A metric measuring how accurately a system identifies which speaker said which words in a multi-speaker recording. Includes missed speech, false alarms, and speaker confusion errors.
Code-switching: The practice of alternating between two or more languages within a single conversation or utterance. A critical failure point for ASR models trained primarily on monolingual data.
Latency budget: The total allowable delay from a user's speech to a system's response, split across transcription, LLM inference, and text-to-speech components. For voice agents, the practical ceiling is approximately 500ms total.
WebSocket: A persistent, full-duplex communication protocol used to stream audio data from a client to a server and receive text output in near-real-time. The transport layer for our real-time STT API.
Voice Activity Detection (VAD): A component that detects whether audio contains human speech or non-speech (silence, background noise). Prevents inference on empty audio frames, reducing both cost and hallucination risk.
Async (batch) transcription: Transcription of a complete pre-recorded audio file, processed after the recording is complete. Provides higher accuracy than real-time because the model has full audio context before generating output.
Data residency: The requirement that data remains stored and processed within a specific geographic region. Relevant for GDPR compliance in the EU and data sovereignty policies in regulated industries.
