Most engineering teams build a real-time transcription path only to discover their downstream LLM prompts are failing because of fragmented, out-of-order WebSocket packets. The transcription layer looked fine in staging. Production exposed the gap. This guide is the architectural blueprint for avoiding that outcome: how to layer real-time streaming onto an existing asynchronous STT pipeline without touching the core data path, how to manage the WebSocket connection lifecycle, and how to keep the latency budget intact from first audio byte to final committed transcript.
Why integrate streaming into your transcription flow
Critical drivers for streaming STT
Post-call async transcription handles the analytics workload well: CRM population, coaching scorecards, compliance tagging, and summary generation all tolerate a few seconds of processing delay. The case for real-time streaming is narrower but unavoidable once you need it. Live agent assist requires sub-second transcript delivery before the caller finishes speaking. Real-time compliance alerts for PCI (Payment Card Industry) flagged keywords require the same. Interactive voice agents need the transcription layer to feed an LLM pipeline within a latency budget that typically leaves no room for batch processing. Solaria-1 handles 100+ languages with native code-switching at real-time latency, which matters for contact centers serving multilingual customer bases.
The decision to add real-time streaming is typically driven by product requirements that cannot tolerate batch processing delay: live in-call UI feedback, agent coaching overlays, and voice agent response latency constraints are the most common examples.
Architectural tradeoffs for real-time STT
The core tradeoff is accuracy against latency. Async models see the full recording before emitting a final transcript, which gives the diarization pipeline full context, lets the model resolve ambiguous phonemes using downstream words, and enables higher-quality named entity extraction. Real-time models make incremental decisions on incoming audio windows and commit earlier, which means more corrections to partial results and no full-context diarization.
Table 1: Build vs. buy decision matrix
| Metric |
Self-hosted open-source |
Gladia managed API |
| GPU maintenance |
Dedicated cluster, ongoing DevOps* |
Managed infrastructure, no GPU required |
| Engineering hours |
20%+ sprint capacity on infra ops |
Sub-24-hour integration (customer-reported) |
| Latency |
Varies widely. Sub-300ms requires quantization, custom batching, or dedicated inference server configuration |
Sub-103ms partials, approximately 300ms end-to-end final (Solaria-1) |
| Total cost of ownership |
GPU cluster provisioning, ongoing DevOps labor, network egress, and model maintenance. Total cost of ownership often exceeds a managed API before production scale* |
From $0.25/hr real-time, $0.20/hr async (Growth) |
*Self-hosting open-source STT models trades one cost for another. Eliminating vendor fees introduces GPU cluster provisioning, ongoing DevOps labor, network egress, and model maintenance. These costs can exceed a managed API well before you reach production scale. Teams that run self-hosted open-source models often report over 10% word error rate on production audio once you account for the absence of continual model updates and GPU instability under concurrent load.
WebSocket integration patterns for live audio
Connection lifecycle and retry logic
A client-side audio buffer queues frames during transient disconnects and flushes them on reconnect, part of the full connection lifecycle the live WebSocket API defines. Implement connection health monitoring with periodic keep-alive checks, reconnecting when the connection degrades or times out. Persist the session ID on the client so the downstream queue can correlate segments from reconnected sessions into the same conversation record.
For retry logic, implement standard exponential backoff with a reasonable cap and retry limit. After exhausting retries, signal the downstream application that the stream has failed permanently and route the buffered audio directly to the async pipeline via Solaria-3. This fallback keeps the conversation record complete even when the live layer fails, which matters for compliance logging regardless of whether the real-time UI feature delivered.
Encoding requirements for live streams
The live STT getting-started documentation requires the initial configuration message to declare encoding, sample_rate, bit_depth, and channels before any audio frames are sent. A commonly used configuration is 16kHz sample rate, 16-bit depth, mono. Ensure the declared configuration matches what the client actually sends to avoid transcription errors during integration.
Audio should be sent in small, continuous chunks sized for low-latency delivery. For containerless formats like raw PCM, strip headers before sending over the WebSocket.
Audio format reference:
| Format |
Sample rate |
Bit depth |
Notes |
| PCM |
16000 Hz |
16-bit |
No codec processing overhead. Highest bitrate, use when bandwidth is not a constraint |
| WAV |
16000 Hz |
16-bit |
Include header on first frame |
| Mu-law |
8000 Hz |
8-bit |
Common in telephony (SIP/RTP), can be sent raw or in RTP wrappers |
Efficient auth for streaming sessions
The initial configuration message must include x-gladia-key as the authentication credential, sourced from your API key at app.gladia.io. In production, never embed the API key in client-side code. Route the WebSocket initiation through a backend proxy that issues short-lived tokens. This pattern also lets you enforce per-session rate limits before the connection reaches the transcription layer.
On Growth and Enterprise plans, customer audio is never used for model training by default, with no opt-out required. On the Starter plan, data can be used for training by default. The full compliance posture (SOC 2 Type II, GDPR, ISO 27001, HIPAA) is documented at our compliance hub, including GDPR and PCI DSS specifics for contact center deployments.
Handling transcript events in streaming architectures
Partial results: sub-103ms transcription fragments
The partial transcripts documentation distinguishes two event types: partial (interim, subject to revision) and final (committed, stable). Partial events arrive under 103ms and represent the model's current hypothesis. Final events replace the preceding partials with the committed transcript segment once the model has sufficient lookahead context. A common UI pattern is to render partials in a lower-confidence style, then replace with the final text on the committed event. Downstream processing, such as LLM calls, CRM writes, and analytics events, should trigger only on final events. Partial updates to the UI display layer are expected and supported as the model refines its hypothesis.
Determining production commit thresholds
The model commits a segment as final primarily through silence detection: once the audio stream falls silent beyond the endpointing threshold, the current utterance is treated as complete and the segment is committed. You can influence commit cadence by adjusting the endpointing parameter, which sets the silence duration threshold before a turn is treated as complete. For conversational telephony audio with frequent speaker overlaps, raising this value reduces premature commits that cut off the tail of a speaker's utterance. The audio chunk acknowledge reference describes the ack signal pattern that supports client-side tracking of which chunks have been confirmed.
Deduplicating real-time transcription data
In streaming systems, reconnects can sometimes cause duplicate events. To prevent duplicate utterances from appearing in your CRM or LLM input, implement deduplication logic on the client side by tracking which segments have already been processed before passing events to the downstream queue.
Latency budget: end-to-end and partial targets
Measuring end-to-end latency in production
End-to-end latency for a streaming STT pipeline has several components, and you need to instrument each one separately to find the bottleneck when targets are exceeded:
- Audio capture and chunking: Client-side recording, format conversion, and chunk assembly.
- Network transport: Round-trip time (RTT) from client to our nearest regional endpoint.
- Model inference: Partial results arrive in under 103ms, final segment commitment (end-to-end) targets approximately 300ms. Both figures apply to Solaria-1 under normal network conditions.
- Downstream application processing: LLM inference, database writes, and UI push (budget the majority of your remaining latency window here). For a live agent assist use case with a tight total latency budget, the downstream LLM call is almost always the limiting factor once STT is tuned correctly.
Managing network jitter and audio buffers
A client-side jitter buffer smooths packet arrival variance without pushing total latency past the budget for most LAN and low-latency WAN environments. Jitter buffers in streaming systems generally reorder incoming packets before delivering them to the application layer. If your client observes out-of-order chunk delivery, resequencing by chunk index before passing data downstream is the standard mitigation. Packet reordering also contributes to jitter, though in practice it is rare outside of NACK retransmission scenarios. The buffer provides enough audio context for proper turn-taking detection, though distinguishing a brief pause within an utterance from a genuine turn end requires advanced turn-taking models that analyze semantic content, not just silence duration.
Common causes of latency overruns in streaming pipelines include:
- Oversized audio chunks: Streaming models have an intrinsic latency equal to the duration of a single chunk. The model cannot begin processing until the full chunk is received. For real-time pipelines, chunk duration should be kept as small as practical, since larger chunks introduce proportionally higher intrinsic latency before any inference begins.
- Misconfigured sample rate: A mismatch between declared and actual sample rate forces resampling and adds latency.
- Slow downstream LLM inference: STT arrives on time, but the LLM call blocks the thread, making the pipeline appear slow.
- Load balancer timeout misconfiguration: Many cloud load balancers default to short idle timeouts that close the WebSocket connection during natural pauses in conversation. Increase this for telephony workloads.
Optimizing audio buffering for turn-taking logic
Configuring VAD and managing speaker shifts
Voice Activity Detection (VAD) configuration controls when the model treats a pause as a turn boundary and commits a final segment. The endpointing parameter sets the silence duration threshold. Tune this value to match your audio profile: shorter values for voice agents that need to respond quickly, longer values for telephony audio with frequent filler words and short pauses. Adaptive VAD adjusts silence thresholds to the ambient noise floor, reducing false triggers in noisy call center environments compared to a fixed threshold.
Silence, background noise, and speaker transitions all produce edge cases in the stream. Gaps longer than the endpointing threshold trigger a final commit and a new segment, which is the correct behavior for clean turn-taking. For overlapping speakers, the model commits the dominant speaker's audio and treats the overlapping speech as background noise during that window.
Speaker diarization, which attributes committed segments to named speakers, is powered by pyannoteAI's Precision-2 model and is only available in asynchronous workflows. For hybrid pipelines, speaker attribution should be applied in post-processing after the session closes and the full audio is submitted to Solaria-3. The speaker diarization documentation describes the async diarization configuration in detail.
Closing active streams without data loss
The teardown sequence for a WebSocket session matters as much as the connection setup. The correct sequence is:
- Client sends the end-of-stream signal per our WebSocket protocol.
- Server flushes any buffered audio and emits the final transcript event for the trailing segment.
- Server sends the WebSocket close frame.
- Client confirms receipt and logs the session ID with final metrics.
Closing the connection without sending the end-of-stream signal causes trailing audio to be discarded, producing a consistent gap at the end of every transcript, which is a compliance problem for call recording workflows where every word matters.
Adding streaming layers without pipeline overhauls
Running async and streamed pipelines
The hybrid architecture bifurcates the data flow at the audio capture layer, not at the transcription layer, and both paths start from the same raw audio stream:
[Client Audio Capture]
|
______________________↓____________________________
↓ ↓
[WebSocket] [Raw Audio Buffer → Object Storage]
↓ ↓
[Solaria-1] [Session Close Event → Async Queue]
↓ ↓
[Partials → UI] [Solaria-3 + Diarization + Named Entity Recognition]
↓
[LLM Pipeline / CRM / Analytics]
The WebSocket stream feeds Solaria-1 for partials and final segment commits that drive the live UI. Simultaneously, raw audio frames are written to a buffer and submitted to the Solaria-3 async API when the session closes. The audio-to-LLM documentation describes how to structure the post-session submission for LLM-ready output with diarization, named entity recognition, and sentiment included.
Core WebSocket and buffering logic
The key pattern is handling both the WebSocket send and the buffer write in the same audio chunk handler, eliminating coordination overhead between the two paths:
```javascript
// Initialize WebSocket connection and audio buffer
// Requires the `ws` package: npm install ws
const WebSocket = require('ws');
const ws = new WebSocket(gladiaWsUrl); // Replace with your Gladia live session WebSocket URL
const audioBuffer = [];
const queue = []; // Queue for final transcript segments
ws.on('message', (raw) => {
const message = JSON.parse(raw);
if (message.type === 'transcript' && message.data.is_final) {
queue.push({
segmentId: message.data.id,
text: message.data.utterance.text,
});
}
});
// audioCapture represents your audio capture module
audioCapture.on('chunk', (chunk) => {
ws.send(chunk); // real-time path
audioBuffer.push(chunk); // async path buffer
});
// session represents your session management module
session.on('close', () => {
const fullAudio = Buffer.concat(audioBuffer);
// gladiaAsyncApi represents your Gladia async API client
gladiaAsyncApi.submit(fullAudio, { model: 'solaria-3', diarization: true });
audioBuffer.length = 0;
});
```
If the WebSocket connection drops, the buffer continues uninterrupted, and the async submission recovers the full conversation record regardless of streaming failures.
Controlling feature flag migrations and observability
Roll out the streaming layer behind a feature flag gated at the session level, not the user level. This lets you activate real-time for a specific call queue, agent group, or tenant before broad exposure. The async path continues to run for all sessions regardless of flag state, which keeps the analytics pipeline intact during rollout and gives you a clean quality comparison between both paths on the same calls. Rollback is a single flag toggle, automated off your monitoring threshold for sustained connection failure rate.
Track four metrics per session to cover both paths:
- WebSocket connection duration and failure rate: Prometheus counter and gauge via the
close and error events. - Partial result latency p50/p99: Time between sending the audio chunk and receiving the corresponding partial event (p50 = 50th percentile, p99 = 99th percentile).
- Async submission queue depth: Monitor lag in your message queue (SQS, Kafka, RabbitMQ, etc.) for the post-call processing path.
- End-to-end pipeline latency: Time from session close to final CRM write, covering both async transcription and LLM inference. Dashboarding both paths in the same Datadog or Prometheus namespace gives you a single view for incident triage rather than two separate systems to cross-reference during an outage.
When to select streaming over batch processing
WER benchmark methodology
Table 2: Benchmark methodology disclosure
All providers tested on identical audio files via production APIs with default settings.
| Dataset |
Audio condition |
Solaria-1 WER |
Solaria-3 WER |
| Earnings22 (financial calls) |
Business speech, varied accents |
8.1% |
6.4% |
| Switchboard (conversational) |
Challenging telephone audio, natural speech* |
37.3% |
33.9% |
*Switchboard is a particularly challenging conversational telephone dataset with cross-talk, background noise, and informal speech patterns, resulting in higher WER across all providers tested.
Both models were benchmarked against AssemblyAI, ElevenLabs, Deepgram, Mistral, and Speechmatics on the same audio files under identical conditions. On Earnings22, Solaria-3 is the only model under 7% WER across all providers tested; on Switchboard, it is the only model under 35%. Solaria-1 is separately benchmarked against eight providers across seven datasets and 74+ hours of audio, where it posts on average 29% lower WER on conversational speech than the tested alternatives.
For streaming workloads, Solaria-1 handles noisy call center audio and code-switching natively at real-time latency. Solaria-3 is the right model for post-call analytics on European business audio, where its improved accuracy over Solaria-1 on real English production recordings, as shown in Table 2, justifies the async processing delay.
Cost per hour: streaming vs. async
On the Growth plan, async transcription with Solaria-3 runs as low as $0.20/hr and real-time streaming with Solaria-1 runs as low as $0.25/hr. Both rates include diarization, translation, sentiment analysis, named entity recognition, and summarization with no add-on fees. Full pricing on our pricing page.
At 10,000 hours per month of contact center audio, a hybrid pipeline that streams all calls for live assist and processes all calls async for post-call analytics costs approximately $4,500 per month at the Growth plan's committed-volume floor rate ($2,500 streaming + $2,000 async. The "as low as" figures require upfront volume commitment, entry rates are higher). The CCaaS use case page maps both patterns against specific contact center product requirements.
When to prioritize real-time streams
Use streaming when the product requires any of these: live agent assist overlays, real-time compliance keyword alerting, voice agent response latency constraints, or live captioning for active participants. Use async-only when the product is entirely post-call: summaries, coaching scorecards, CRM population, and quality assurance workflows. The blind STT comparison tool strips out provider branding so you can pick the better transcript before seeing which model produced it, though follow it with a reproducible benchmark on your full audio distribution before making a production commitment.
Troubleshooting your streaming pipeline
Latency and connection drop diagnostics
When latency exceeds targets in production, work through this checklist in order:
- Verify chunk size: Confirm chunks are small and continuous. Oversized chunks introduce batching delays that inflate latency beyond the 300ms target.
- Check RTT to the regional endpoint: Route to whichever of our EU or US regional endpoints is closer to your client deployment.
- Validate audio configuration match: a mismatch between the declared configuration (
sample_rate, encoding, bit_depth, channels) and what the client actually sends will produce garbled transcripts. Confirm all four match exactly before debugging anything else. - Monitor WebSocket health: RFC 6455 ping/pong frames indicate connection health independently of transcript delivery. A pong latency threshold for triggering reconnect is a configurable implementation detail. Set it based on your latency budget, not a fixed standard. Frequent drops are usually caused by one of three infrastructure issues: load balancer idle timeout set too short, network firewall blocking WebSocket upgrade headers, or client-side memory pressure causing the WebSocket thread to pause. For the firewall case, verify that
Upgrade: websocket and Connection: Upgrade headers pass through the proxy layer unmodified, as RFC 6455 requires.
Start with €50 in free credits and have your WebSocket integration in production in under a day.
FAQs
Can I run streaming and async in the same pipeline?
Yes. Stream live audio to Solaria-1 for immediate UI feedback while simultaneously buffering the raw audio and submitting it to Solaria-3 after the session closes for high-accuracy post-call analytics, including diarization and named entity recognition. The two paths are fully independent, so a failure on the streaming path does not affect the async submission.
What is the latency of Gladia's real-time transcription?
Solaria-1 delivers partial results in under 103ms and final transcripts with an end-to-end latency of approximately 300ms. Real-time streaming runs on Solaria-1 exclusively, as Solaria-3 is an async-only model.
Does Gladia support on-premises deployment?
No. We do not support on-premises or air-gapped hosting on any plan and deploy exclusively to dedicated cloud clusters in EU and US regions.
Is speaker diarization available in real-time streams?
No. Speaker diarization is powered by pyannoteAI's Precision-2 model and is strictly limited to asynchronous workflows, because full-context diarization requires the complete recording before clustering algorithms can assign speaker labels accurately. In a hybrid pipeline, diarization runs on the Solaria-3 async path after the session closes.
What audio formats does the WebSocket streaming API accept?
The primary formats for WebSocket streaming are raw PCM at 16kHz/16-bit and Mu-law at 8kHz/8-bit. WAV is supported when the header is included on the first frame. The initial configuration message must declare encoding, sample_rate, bit_depth, and channels before any audio frames are sent, and the declared values must match what the client actually streams.
What happens to the transcript if the WebSocket drops mid-call?
Committed transcript segments are preserved client-side. Uncommitted audio in the ring buffer is retained for the reconnect window. If reconnect fails permanently, the buffered raw audio submitted to Solaria-3 on the async path recovers the full conversation record, including segments that were in-flight when the connection dropped.
Key terms glossary
Word error rate (WER): The standard metric for speech-to-text accuracy, calculated by dividing the total number of insertions, deletions, and substitutions by the total words spoken. Lower is better.
Voice Activity Detection (VAD): An algorithm that detects the presence or absence of human speech in an audio stream, used to determine turn boundaries and trigger final segment commits in streaming pipelines.
Partial results: Interim transcription fragments returned by the model as it processes incoming audio chunks, updated dynamically as the model refines its hypothesis before emitting a final committed segment.
Diarization Error Rate (DER): The metric for speaker diarization accuracy, measuring the percentage of audio time not correctly attributed, including speaker confusion, missed speech, and false alarms. Available in async workflows only.
Code-switching: The practice of alternating between two or more languages within a single conversation, supported natively in Solaria-1 across all supported languages.
Endpointing: The silence duration parameter that controls when the streaming model treats a pause as a turn boundary and commits a final segment. Check the live STT documentation for the current default value before configuring for production.
Ring buffer: A fixed-size circular buffer used on the client side to queue audio frames during transient WebSocket disconnects, enabling session continuity on reconnect without data loss.
