Speech-to-Text Benchmark Results - Word Error Rate (WER %, lower is better)

Methodology: benchmarks run on the same framework as the Solaria-1 campaign. Solaria-3 and Solaria-1 compared against 8 speech-to-text providers. Each provider tested on identical audio files via production APIs with default settings. Real customer audio = Gladia internal production dataset, annotated by humans. Soniox and Pipecat STT Benchmark excluded on some datasets pending data availability. Transcriptions normalized with OpenAI Whisper text normalizer before WER computation. Diarization Error Rate (DER) measured on DIHARD III using standard protocols. 74+ hours of audio across 8 evaluation datasets. Full open benchmark methodology: https://github.com/gladiaio/normalization

Columns: WER = Word Error Rate (%, lower is better) · Perfect = number of files with 0% WER · High WER = files with WER above 50% · RTFx = Real-Time Factor multiplier (higher = faster than real-time)

Real Customer Audio - English (Gladia Internal Dataset)

Switchboard - Conversational Speech

Common Voice 24 - Clean Audio (Multilingual)

VoxPopuli - Formal Discourse (European Parliament)

Earnings22 Full - Financial Calls (Long-form, single file)

Earnings22 Cleaned AA - Financial Calls (Curated by Artificial Analysis)

Multilingual LibriSpeech - Audiobooks (5 languages, average)

Multilingual LibriSpeech - WER by Language (%)

Pipecat STT Benchmark - Real-Time Streaming

Speaker Diarization Benchmark - DIHARD III - Diarization Error Rate (DER %, lower is better)

Speech-to-text benchmark FAQs

Which speech-to-text API has the lowest word error rate?

It depends on the audio. In this benchmark, Gladia's Solaria-3 has the lowest word error rate on real customer calls (9.6%), Switchboard telephone conversations (33.9%) and Earnings22 cleaned financial calls (6.4%). Speechmatics leads on Common Voice 24 (3.8%), ElevenLabs Scribe v2 leads on VoxPopuli (1.7%) and Multilingual LibriSpeech (3.7%), and AssemblyAI Universal v3 leads on the Pipecat real-time streaming benchmark (2.0%). Test on audio that matches your production use case.

How is word error rate measured in this benchmark?

Word error rate is the share of words a model substitutes, deletes or inserts compared with a human reference transcript. Each audio file is sent to every provider's production API with default settings, and every output is normalized with the OpenAI Whisper text normalizer before scoring. The full framework is open source on GitHub so anyone can reproduce the results.

What does RTFx mean in speech-to-text benchmarks?

RTFx is the real-time factor multiplier: how many times faster than real time a provider transcribes audio. Higher is faster. On the Earnings22 cleaned dataset, for example, Deepgram Nova-3 runs at 234x and AssemblyAI at 64x. RTFx measures throughput, not the delay before the first words appear in a live stream.

Which speech-to-text API is most accurate for speaker diarization?

Diarization accuracy is measured with diarization error rate (DER), where lower is better. On the DIHARD III benchmark, Gladia's Solaria-1 has a weighted average DER of 16.6%, compared with 20.4% for NVIDIA NeMo Sortformer, 23.0% for pyannoteAI Community-1 and 30.1% for Speechmatics.