So how does one spot and handle multilingual transcription errors before they spread downstream? We made this guide to answer that question, showing how to run tests yourself, understand why code-switching and real-time streaming make it harder, and figure out what to do when one of your languages falls short.
Key highlight: Multilingual transcription accuracy depends more on the audio than on the language: every speech-to-text model scores far worse on real calls than on read speech, so each language should be tested on production audio. In Gladia’s benchmarks, run on the same files with one normalizer, Solaria-3 ranks #1 among major providers on real customer calls. Gladia’s Solaria-1 covers 100+ languages with code-switching, in real time and async.
What is multilingual transcription?
Multilingual transcription is the ability to turn speech into text when more than one language is involved. That can mean transcribing recordings in individual languages, or a single conversation where several languages are spoken.
It’s easy to confuse with translation, but translation comes later. First you transcribe, then you translate the transcript into whatever language each reader needs. That order matters, because a translation can’t fix what the transcript got wrong.
So where does all this multilingual audio come from? Mostly client calls, meetings, research interviews and videos with speakers from different countries. And there’s more of it every year: WiseGuy Reports expects the multilingual transcription service market to more than double, from $2.62 billion in 2024 to $6.0 billion by 2035.
Some of that audio, like recordings with legal weight, will always go to human transcribers. Most of it, though, now runs through speech-to-text APIs, with people checking only the parts that matter. That’s why the model’s accuracy matters so much: it decides how much of that checking your team has to do.
Why multilingual transcription is harder than English-only
Most speech-to-text models are benchmarked on clean English, but real multilingual audio rarely looks like that. Speakers have accents, there’s background noise, and people switch languages halfway through a sentence. Four things explain most of the accuracy gap:
Training data is uneven across languages
A model only learns a language as well as its training data covers it, and that coverage is far from equal. English, Spanish, French and German have huge public datasets behind them, while Tagalog, Bengali, Javanese and Haitian Creole have a fraction of that. OpenAI saw the effect in its own Whisper paper (2022), where word error rate (WER) varied widely by language, and languages with less training data scored worse.
One shortcut to more data for under-covered languages is to train on machine-labeled audio, meaning transcripts produced by another model instead of people. Whisper large-v3 (November 2023) did this at scale, adding 4 million hours of audio transcribed by Whisper large-v2. The learning: machine-labeled data copies the labeling model’s mistakes, from broken punctuation to phrases hallucinated over silence.
That’s why open projects have since tackled the gap more carefully, with recordings from native speakers or with machine labels that get filtered before training:
| Project |
Released |
Coverage |
What it adds |
| Mozilla Common Voice |
Ongoing, v26 in June 2026 |
294 languages of read speech, 78 of spontaneous speech |
Crowdsourced, openly licensed recordings; a code-switching dataset is in alpha |
| NVIDIA Granary |
2025 |
25 European languages, about 643,000 hours for speech recognition |
Machine-labeled audio with hallucination filtering, including Croatian, Estonian and Maltese |
| African Next Voices |
2025 |
9,000+ hours across 18 languages in Kenya, Nigeria and South Africa |
Community-recorded speech on everyday topics such as health and agriculture |
| Meta Omnilingual ASR |
November 2025 |
1,600+ languages, 500 of them never served by ASR before |
Open models from 300M to 7B parameters that can add a language from a handful of samples |
| Google WAXAL |
2026 |
24 Sub-Saharan African languages, 14,100+ hours |
Natural speech prompted by images, about 2,340 hours of it transcribed |
Still, more coverage doesn’t automatically mean more accuracy. Even Omnilingual ASR, built specifically for rare languages, keeps its character error rate below 10 for 78% of its languages. For the other 22% of languages, it gets more than one character in ten wrong. So a language can appear on a model’s supported list even though the model hasn’t heard enough of it to transcribe it well. The only way to tell “supported” from “trained well” is to test that language on your own audio.
Accents and dialects vary inside one language
The same goes for accents, even within one language. A model that handles standard British English well can still stumble over a Glaswegian or Mumbai accent, because it only learns the voices it was trained on. And most of that training audio is clearly spoken, standard US English. We cover regional accents and speakers who mix languages hardest in more detail in the language bias in ASR article.
The speaker’s profile matters too, not just their accent. In that same article, our CEO points out that most of the abundant audio online comes from a similar source: a male content creator in a quiet room with a good headset. In his experience, that skew shows up as accuracy above 90% for men, around 80% for women and as low as 40% for children. Independent research paints a more mixed picture, though. A 2024 study of Whisper and SeamlessM4T across 19 languages found gender gaps in most of them, but which group came out ahead changed from language to language. Either way, the takeaway is the same: split your test set by speaker group as well as by language.
Domain vocabulary breaks first in low-resource languages
Combine thin training data with specialist terms, and that’s usually where transcripts break first. A model that copes fine with everyday Tagalog can still miss Tagalog medical terms or product names, simply because it has rarely heard them. This is where custom vocabulary helps. You give the model a list of terms to expect for each language, and it spells them consistently across the transcript.
Errors land on specific users, then flow downstream
The tricky part is that these errors don’t spread evenly. They pile up on the customers who speak the less common language, have a stronger accent or mix languages. So your average WER can look healthy while one group of users gets transcripts nobody can use. And because CRM fields, QA scores and summaries are built on those transcripts, every mistake travels with them. Code-switching is where this shows up most. When a speaker changes language mid-call, the transcript usually fails in one of three ways:
- Dropped words: the second language comes back missing, or as unknown tokens.
- Made-up text: the model hears the second language and writes fluent first-language words nobody said.
- Silent gaps: the whole switched section vanishes, with no sign of it in the transcript.
How to measure speech-to-text accuracy per language
Pick WER or CER by writing system
The right metric depends on the language. For most languages, that’s WER (the share of words a model gets wrong), measured one language at a time on audio that looks like production. Chinese and Japanese don’t put spaces between words, though, so there you’d use CER (character error rate, the same calculation at character level), which suits Thai too. And if your transcripts feed an LLM, MER (match error rate) is a useful second metric, because it weighs insertions differently. That per-language approach is what makes multilingual transcription accuracy comparable in the first place.
Normalize text before you score
Before you compare any numbers, put every transcript into the same written form. Otherwise numbers, punctuation and casing can move WER by several points without a single real recognition error. “Twenty-one” and “21”, for example, count as a mismatch unless both are normalized. So use one normalizer for every provider and for your reference transcripts. Each language has its own rules, and our normalization guide for French, German and Spanish covers them in detail. There’s a catch, however: Normalization strips punctuation and casing before scoring, so a model can post a near-perfect WER on transcripts that are hard to read. Our CEO calls it “perfect WER, dirty readability.” If people or an LLM will read your transcripts, check punctuation and casing on a sample as well.
Build a per-language test set from your own audio
Published benchmarks can’t tell you how a model will handle your calls, but a test set built from your own recordings can. It’s the only one that predicts production accuracy, and it takes less time to set up than most teams expect:
- Pull real audio for each language and get human reference transcripts made. Keep the code-switched clips as their own set, so they don’t get averaged away.
- Leave the mess in. Noisy audio, multiple speakers talking over each other and your own domain terms are what production looks like. Clear audio flatters every model.
- Give every provider the same treatment: identical settings and the same normalizer, or you’re comparing formatting instead of accuracy.
- Score with JiWER, an open-source Python package that calculates WER, MER and CER.
- Report each language on its own, and start with your lowest-resource one. That’s usually where providers pull apart.
Read published benchmarks for the dataset, not the headline
A WER figure means little until you know the dataset behind it and how the text was normalized. FLEURS, Common Voice and Multilingual LibriSpeech, for example, are all read speech, so they tell you little about a phone call with people talking over each other. It’s also worth checking that the evaluation code is public, so you can rerun the numbers yourself. We hold our own open benchmark to that standard.
How code-switching and real-time streaming affect multilingual accuracy
Even a model that scores well on your per-language test set can lose accuracy in two situations that come up often in production. One is when speakers switch languages mid-conversation, and the other is when the transcript has to arrive in real time.
Code-switching needs per-utterance language detection
Take “La onda is to fight y jambar”: two language switches in six words. A model that decodes one language at a time can’t keep up with that. It needs to detect the language for every utterance.
| What the speaker said |
Single-language decoding |
Per-utterance detection |
| “La onda is to fight y jambar” |
Spanish words dropped or rewritten as English sound-alikes |
Each segment transcribed in the language spoken, with a language code |
In this case, two settings mix most of this. First, have the model detect the language for each utterance, not just once at the start of the file and then assume it for the rest. Second, tell the model which languages to expect. Gladia’s code-switching docs specifically warn against turning on code-switching with an empty language list, since the detector then has to weigh every utterance against 100+ languages.
Async beats real-time on multilingual accuracy, with exceptions
If you can wait for the transcript, wait. Async multilingual transcription is usually more accurate, because the model hears the whole recording before deciding which spoken language each part is in. A streaming model has to make that call from a few hundred milliseconds of audio. You’ll see the difference at language boundaries, in named entity recognition and in speaker diarization. Speaker labels and word-level timestamps are what turn accurate transcripts into something a team can work with.
When real-time is the right call
Real-time is the right call when you need the text during the conversation, as with voice agents, live captions and agent assist. That’s why several providers build dedicated streaming models, such as Deepgram’s Flux and AssemblyAI’s Universal-3.6 Pro Realtime. Meeting notes, post-call analytics and video don’t need that speed, so async is the better pick there.
If you do stream, tell the model which languages to expect rather than leaving detection open. Gladia’s real-time API runs on Solaria-1, with partial transcripts in under 103 ms, and our ML team explains how they cut the lag after each language switch in building real-time multilingual ASR with code-switching.
Gladia's accuracy by language and audio type
We publish per-language WER on dedicated language pages, and the table below gathers the results for German, Spanish, French and Italian for our latest Solaria-3 model. Each language is scored on three kinds of audio (lower is better).
| Language |
Real customer calls |
Common Voice (accented read speech) |
FLEURS (read speech) |
| German |
17.2% |
6.3% |
4% |
| Spanish |
15.5% |
5.5% |
3% |
| French |
18% |
8.7% |
4.4% |
| Italian |
20% |
6.1% |
1.8% |
The three datasets range from real-life to read speech. “Real customer calls” is our internal production dataset, annotated by humans. Common Voice has volunteers reading aloud with a wide range of accents and microphones. FLEURS is read sentences, with every language tested on the same content.
What the numbers show
Three patterns stand out, and the first one is the reason we recommend you test on your own audio:
- Audio type moves WER more than language does: On FLEURS, the four languages sit between 1.8% and 4.4%. On customer calls they jump to between 15.5% and 20%, roughly four to eleven times higher.
- Languages swap places between datasets: Italian has the lowest WER on FLEURS and the highest on customer calls. So a read-speech test would mark Italian as your safest language, and production would prove it wrong. Spanish is the steadiest of the four: lowest on calls and Common Voice, and second-lowest on FLEURS.
- Accents cost points even when the text is read: Common Voice is still read speech, but its mix of accents and microphones adds 2.3 to 4.3 points over FLEURS. French and Italian show the widest gap.
Customer-call WER also depends on whose calls they are, since a contact-center line, a meeting app and a voice agent differ in channel, noise and vocabulary. Claap, for example, measured a WER between 1% and 3% on its own meeting audio. That’s a different test set, so it doesn’t compare directly with the table, it does show how much your own audio can move the number.
Need a language beyond these four? Our languages page covers all 100+ that Gladia supports.
Supported languages, features, and price at a glance
Accuracy is only half the evaluation. The other half is what each model supports and what it costs. The table below shows what each provider lists for its multilingual models, checked on their own pricing pages and docs.
| Provider (model) |
Languages |
Code-switching |
Real-time |
Speaker diarization |
List price per hour (batch / real-time) |
| Gladia (Solaria-1) |
100+ |
All supported languages, async and real-time |
Yes |
Async only |
0.27€ / 0.68€ |
| Gladia (Solaria-3) |
Optimized for EN, FR, DE, ES, IT |
Limited |
Async only |
Async |
0.16€ / not available |
| AssemblyAI (Universal-3.5 Pro, Universal-3.6 Pro Realtime) |
99+ with Universal-2 fallback |
18 languages in batch, 32 in real-time |
Yes |
Yes |
$0.21 / $0.45 |
| Deepgram (Nova-3) |
45+ |
10 languages (multi mode) |
Yes |
Included in batch, add-on in streaming |
$0.31 / $0.35 (promotional; $0.55 regular) |
| ElevenLabs (Scribe v2) |
90+ |
Multiple languages per file in batch, with automatic detection; not documented for Scribe v2 Realtime |
Yes (Scribe v2 Realtime) |
Batch only |
$0.22 / $0.39 |
| Speechmatics (Melia 1) |
56 |
All 56 languages in batch; in real-time, seven bilingual packs on Standard and Enhanced |
Yes on Standard and Enhanced; Melia 1 real-time announced, not yet available |
Yes |
$0.12 / not available for Melia 1 |
| Mistral (Voxtral Transcribe 2) |
13 |
Not documented; automatic language detection per request |
Yes (Voxtral Realtime) |
Batch only |
$0.18 / $0.36 |
| Soniox |
60+ |
Automatic, in real-time and async |
Yes |
Included |
$0.10 / $0.12 |
Where Gladia's models fit
We run two models for multilingual transcription, and your languages decide which one fits. Solaria-1 covers 100+ languages in real-time and async. Solaria-3 is async-only and built for core European languages.
|
Solaria-1 |
Solaria-3 |
| Languages |
100+ languages |
Optimized for English, French, German, Spanish and Italian |
| Code-switching |
Supported |
Limited |
| Real-time streaming |
Yes (partials under 103 ms) |
Async only, for now |
| Best on |
Broad language coverage, read and formal speech |
European business audio, calls, noisy recordings |
On real customer audio, Solaria-3 cuts WER relative to Solaria-1 by 26% in English, 18% in French, 10% in Italian, 9% in Spanish and 3% in German. Solaria-1, on the other hand, remains the better choice on Multilingual LibriSpeech (+36%) and VoxPopuli (+32%). For EU-language depth, see our evaluation of European language speech-to-text coverage and accuracy.
How to improve multilingual transcription accuracy, from settings to fine-tuning
Once per-language testing shows where a model falls short, you have three ways to close the gap. They get slower and more expensive in the order below, so start with the cheapest one that fixes the errors you’re actually seeing.
| Lever |
What it fixes |
What it takes |
| Restrict detection to expected languages |
Segments transcribed in the wrong language, and close pairs such as Ukrainian and Russian resolved in favor of the language with more training data |
A list of expected languages per request |
| Custom vocabulary per language |
Missed names, product terms and jargon |
A short term list, applied at request time with no training |
| Fine-tuning |
Accents, dialects, noisy channels and low-resource languages the base model rarely heard |
Hours of human-verified transcripts in the target condition, plus ML time to train and validate |
Fine-tuning changes what the model hears
Custom vocabulary steers a model toward the right words. Fine-tuning goes further and changes how it hears the sounds in the first place. You keep training a pre-trained model on your own labeled audio until it adapts to an accent, a dialect, a phone channel or a language it barely covered. The gains on under-represented speech can be large: researchers who fine-tuned an open NVIDIA NeMo model on Nigerian-accented English reached 8.2% WER, against 44.2% for the free Google Speech Recognition library on the same test data. Our guide to fine-tuning ASR models walks through the methods, from full-model tuning to LoRA adapters.
That said, a fine-tuning run only pays off with three things in place:
- Human-verified labels: unchecked machine transcripts pass the labeling model’s errors into the new model, the Whisper large-v3 problem described above. Granary’s pipeline filters hallucinations before training for exactly this reason.
- A held-out test set per language: score the tuned model against the base model on audio it never saw, with the same normalizer.
- Regression checks: a model tuned hard on one language or accent can lose accuracy on others, so re-run every language you serve before you ship.
Fine-tuning is also the slowest lever: you need labeled audio you may not have yet, and a tuned model has to be re-validated whenever its base model changes.
On Gladia, expected-language settings and custom vocabulary work on every request. Custom models and fine-tuning come with the Enterprise plan, alongside zero data retention and custom hosting.
Checklist for evaluating multilingual speech-to-text APIs
Before you build a test set, you can use this list of key features to shortlist multilingual transcription software:
- Per-language WER or CER is published, with dataset and date.
- Results include conversational or accented audio.
- Your languages work in the mode you need, real-time or batch.
- Code-switching covers your language pairs in that mode.
- Language detection runs per utterance and can be restricted to expected languages.
- Speaker diarization and custom vocabulary work in your languages, so you can label speakers in every language you transcribe.
- A REST API and real-time endpoint fit your transcription workflow, for any audio or video file and for live streams.
- Compliance and data residency fit your contracts. GDPR and HIPAA apply to most speech transcription involving personal or health data. Gladia holds compliance across GDPR, ISO27001, HIPAA, SOC 2 Type 2, and processes EU and US workloads on separate infrastructure.
- Multilingual transcription cost doesn't change by language.
- Fine-tuning or custom models are available if your weakest language or accent stays out of range after settings and vocabulary changes.
- Code-switched clips included in your own test sets.
Test on your own audio before you pick a provider
A model that scores under 2% on read Italian can score 20% on Italian calls. No published number can tell you how a provider will do on your audience, so pick the one that wins on your languages and your audio, scored per language with one normalizer.
Gladia’s speech-to-text API covers 100+ languages with automatic language detection and code-switching, in real-time and async. We also offer custom fine-tuning for any of these languages, so you can adapt the model to your own audio. Sign up to get €50 of free credit, then run your own multilingual recordings through Solaria-1 and Solaria-3 side by side.
FAQs
Which multilingual speech-to-text API is most accurate across languages?
Multilingual transcription accuracy depends on the languages and the audio, so no single API is most accurate everywhere. Even within one model the spread can be wide: Gladia's Italian WER is 1.8% on FLEURS read speech and 20% on real customer calls. As a first-layer gut check, compare providers on your own recordings with our blind comparison tool.
How does multilingual transcription accuracy differ between major providers?
The gap between providers depends on how hard the audio is. On our benchmarks, run on the same files with one normalizer, WER on real customer calls stays within a few points across providers, from 9.6% to 12.2%. On conversational Switchboard audio it widens to 33.9% to 55.2%. Solaria-3 ranks first on both, and accented and code-switched audio widen the gaps further.
Is batch or real-time multilingual transcription more accurate?
Batch (async) multilingual transcription is usually more accurate, because the model hears the full recording before deciding which language each segment is in. Use real-time transcription when output is needed during the conversation, and batch for meeting notes and post-call analytics.
Which STT API is best for transcribing conversations that switch between languages?
Gladia’s Solaria-1 has the widest code-switching coverage of the APIs compared in this article. It follows language switches across 100+ languages, in both real-time and async, and tags every utterance and word with its language code. Whichever provider you test, check for per-utterance language detection and the option to limit it to the languages you expect.
Which developers provide high-performance speech-to-text APIs for multiple languages?
Gladia, AssemblyAI, Deepgram, ElevenLabs, Mistral, Soniox and Speechmatics all offer multilingual speech-to-text APIs. Rank them by per-language WER on your own audio, not by how many supported languages they list: the same language can score under 5% on read speech and close to 20% on real calls.
How much does multilingual transcription cost?
Multilingual transcription cost varies across transcription services and APIs, and some charge extra for languages or features. On Gladia's pricing, code-switching and all 100+ languages are included in every paid plan, with async transcription from 0.16€ per hour on Solaria-3 and real-time from 0.68€ per hour.
Key terms glossary
Multilingual transcription: software that can transcribe speech in more than one language into text, keeping each segment in the spoken language it was said in.
Word error rate (WER): the share of words a model gets wrong, counting substitutions, deletions and insertions against a reference transcript.
Code-switching: changing languages within a conversation or a single sentence.
Speaker diarization: labeling who said what in a recording with multiple speakers, the basis of speaker identification.
Low-resource language: a language with little transcribed speech available for training, such as Tagalog or Haitian Creole. Models often list it as supported while transcribing it far less accurately.
Pseudo-labeling: transcribing unlabeled audio with an existing model and training on the output. It's cheaper than human labeling but copies the labeling model's errors unless they're filtered, as with Whisper large-v3.
Fine-tuning: continuing to train a pre-trained model on your own labeled audio to adapt it to an accent, a domain or a language.
Text normalization: converting transcripts to a consistent written form so WER compares words, not formatting.
