Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.
Speech-to-text for AI medical scribes: Why clinical vocabulary breaks generic STT
TL;DR: Generic STT engines fail in clinical environments because language model probability overrides correct acoustic detection of medical terms, substituting phonetically plausible but clinically wrong candidates silently. The result corrupts drug names, dosages, and diagnoses before the LLM ever sees them. Before selecting an STT engine for a medical scribe, verify four things: whether vocabulary biasing works at inference time without fine-tuning, whether async diarization accurately separates clinician and patient audio, whether the model holds up on noisy consultation recordings rather than clean read-speech, and whether the vendor's data training policy covers PHI by default on your plan.
Migrating from self-hosted Whisper to a managed speech-to-text API
TL;DR: Self-hosting Whisper's true cost rarely sits in the model weights. GPU idle time, VRAM leaks under parallel load, and the engineering hours spent maintaining CUDA dependencies and diarization pipelines are where the bill compounds. For teams processing under roughly 3,000 hours per month, assuming 20% of one US FTE at $150K loaded annual cost, a managed API is cheaper, though the break-even shifts materially against your actual labor cost. Above that threshold, the decision depends on your DevOps overhead and whether audio accuracy on real-world recordings matters for downstream systems like CRM sync and coaching scores.
Migrating from AssemblyAI to Gladia: A step-by-step switching guide
TL;DR: Switching from AssemblyAI requires four concrete changes: update one auth header, remap batch endpoints, adjust the JSON response schema, and resample audio for WebSocket connections. Multiple customers independently report completing these in under a day with a rollback abstraction layer in place. The bigger structural difference is cost model: a production stack with diarization, sentiment, entities, and summarization runs $0.30/hr on AssemblyAI's Universal-2 tier because each feature is metered separately, versus a bundled base rate. This guide covers the exact parameter mappings, payload diffs, WebSocket reconfiguration, and a zero-downtime cutover strategy.
Using Gladia speech-to-text API with virtual meeting recordings
Published on Oct 10, 2023
Over the past few months, we’ve been working with many of our clients to help them integrate Gladia’s speech-to-text and audio intelligence API into their products.
Many of these companies operate in the virtual meeting space, either by building bots to record and transcribe audio and video meetings on platforms like Zoom, Google Meet, and Microsoft Teams; using these bots to take notes and generate summaries and action items; or by creating their own virtual meeting platform. Together, we’ve identified a few common pain points, challenges, and best practices to overcome them and would like to share hands-on tips on configuring your API to get the best results, including in multilingual environments.
The challenge in virtual meetings
A typical meeting scenario for companies with international or multilingual employees, or for sales and customer support calls, will often start with a few people speaking in one language, usually their native tongue. At some point, one or more additional speakers will join the call, and everyone will switch to a common language from that point onwards.
These scenarios pose a few challenges - correctly detecting the language, understanding who the different speakers are and what each said, and getting results in the required latency.
Recommended configuration
1. Multilingual support
tldr: The best mode for asynchronously transcribing meeting recordings is ‘automatic single language’. It will correctly detect the main language of the meeting and transcribe accordingly while automatically translating any parts spoken in secondary languages.
Gladia supports 3 different modes of language support:
Manual - specify a single language for the audio to be transcribed to.The resulting transcription will be in the specified language, regardless of how many languages were spoken in the audio.Any speech in languages other than the one configured, will be translated automatically. ⚙️ Use for: Cases where a specific pre-known language is needed for the transcript. ⚠️ Limitations: Other languages will be automatically translated to the specified language.
Automatic single language - For asynchronous audio, this behaviour will automatically detect the most prominent language in the audio and use it to transcribe.Segments of speech in other languages will automatically be translated.For live streaming, language will be automatically detected based on the first chunk of audio and will be used for the remainder of the stream. ⚙️ Use for: Most common use cases, including meetings that start in one language and switch to another. ⚠️ Limitations: Secondary languages will be automatically translated to the main language (the one most spoken); Live transcription will detect based on the first utterance only.
Automatic multiple languages (code-switching) - This mode will continuously detect the language spoken and support multiple language changes in a single audio.The resulting transcription will be in multiple languages. ⚙️ Use for: Use cases with languages being changed multiple times in the audio, e.g. a journalist interviewing someone in one language and getting responses in another. ⚠️ Limitations: Currently sensitive to strong accents, which may cause the wrong language to be detected.
2. When to use live vs asynchronous transcription
Tl;dr: For most needs, using async transcription will provide the best balance between price, accuracy, and speed. For cases where real-time or very low latency transcription is needed, use live transcription.
Gladia supports two types of transcription - asynchronous (aka batch) and live. With the asynchronous API, audio files are sent to be transcribed by our AI model, and a transcription is returned in response. Some of the benefits of this method are: the ability to use automatic single language detection (mentioned in the previous section), using a highly accurate transcription model, and being able to use speaker diarization to identify text spoken by different speakers.
With our live transcription API, a WebSocket connection is opened between the user and Gladia. Audio is streamed in small chunks, and transcription results are continuously sent back. This method provides results in real-time with very low latency, and is useful for situations where results need to be displayed quickly in reaction to what’s being said.Since the audio is transcribed in small chunks - language detection with automatic single language will only be based on the first utterance in the audio.
3. Speaker diarization
Tl;dr: When using the API of a virtual meeting service or recording bot, it’s better to rely on their built-in mechanical diarization API, based on separate audio sources. If no such API is available, Gladia provides a high-quality audio-based diarization.
Speaker diarization is the process of assigning the correct speaker to different segments of speech in an audio. Virtual meeting platforms such as Zoom receive the audio from different participants separately, so they usually have an API to correctly assign different speakers to different portions of the audio. When combined with accurate word-level timestamps, this can be used to generate a highly accurate diarized transcript. Most meeting bot providers offer this type of diarization in their API.
In cases where mechanical diarization is unavailable, it’s recommended to use Gladia’s diarization feature, which uses AI and audio analysis to detect multiple speakers in a single audio file and assign the correct speaker to each text segment.
4. Importance of sync and accurate timestamps
Tl;dr: Many advanced features, such as aligning speaker separation and transcription, audio-based diarization, and sending transcriptions to LLMs, require extremely accurate word-level timestamps. Luckily, Gladia provides one of the most accurate timestamp alignment solutions in the market, including for live transcription.
Any form of analysis of a speech recording, especially in a virtual meeting scenario, starts with the accuracy of the transcript timestamps.
Producing an accurately diarized transcript by synchronizing the text to the mechanical audio diarization and understanding which speaker said what, relies on correct and accurate timestamps. Even just a few seconds of incorrect offset could result in a completely wrong diarization, which attributes text to the wrong speaker, changing the meaning and context of what was said. For some use cases such as sales and CRM enrichment, this would make the whole transcript meaningless.
In turn, sending a meeting transcript to an LLM to generate insights such as summaries, chapters, data extraction, requires the utmost accuracy from the diarization. Text that’s attributed to the wrong speaker will generate incorrect and often entirely incoherent insights.
Summary
Navigating the various transcription features and options and finding the optimal configuration for your meeting recording product can be tricky. We’ve gathered the experience from multiple companies with similar needs and challenges and combined the knowledge into the guidelines in this post.We hope it will help you build a better product while significantly shortening the development time.
At Gladia, we built an optimized version of Whisper in the form of an API, adapted to real-life professional use cases and distinguished by exceptional accuracy, speed, extended multilingual capabilities and state-of-the-art features, including speaker diarization and word-level timestamps.
Contact us
Your request has been registered
A problem occurred while submitting the form.
Read more
Speech-To-Text
Medical speech-to-text for AI scribe builders
Speech-To-Text
Migrating from self-hosted Whisper to a managed speech-to-text API
Speech-To-Text
AssemblyAI to Gladia migration guide: API mapping & setup
From audio to knowledge
Subscribe to receive latest news, product updates and curated AI content.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
