Mastering AI transcription for social media captions: Mojo's success story with Gladia

It has been estimated that in 2023 people globally have watched, on average, 17 hours of online video per week (that is, at least one hour daily) — and there’s no reason to expect the trend to subside in the future.

In this context, high-quality automatic captions for videos are becoming a must-have feature for video marketing, given that most consumers expect to be able to consume video on the go without sound. Captions are likewise incredibly important to ensure the accessibility of one’s social applications to all types of users.

Mojo, which specializes in social media content editing, is among our clients relying on speech-to-text to generate high-quality automatic captions and other voice-based editing features.

About Mojo

Mojo is a social video and content app, designed to make high-quality social media content creation easy. With its one-stop-shop app, Mojo users can access hundreds of templates, text effects, and high-quality animations to create stunning social content in a matter of minutes.

Founded in France in 2018, the company targets primarily small businesses in search of easy-to-use tools to showcase and promote their brand across social media channels with Reels, stories, TikTok, posts and more. In the last four years, the app has gained over 46M users, including several hundred thousand paying subscribers.

Challenge

Mojo’s team knew that delivering on its mission meant that they had to make video content editing easy, intuitive, and feature-rich. To stand out in a globalized, ultra-competitive market, the company focused on building a library of hundreds of animated templates and delivering advanced editing tools and effects, such as video trimming and background removal.

The integration of audio transcription in particular into Mojo's app came in response to growing user demand for auto-captions, aligning with the company’s ongoing shift from template-based to tool-based app usage.

Having tried several alternative speech-to-text APIs, Mojo’s team turned to Gladia in search of a better solution.

The main issue encountered with the incumbent provider was that it did not meet the quality standard in word timings, aka word-level timestamps, which were especially critical for the right timing of visual effects.

In addition, the language support of the previous provider was not always reliable to the detriment of client satisfaction and retention.

Objectives

To deploy a high-quality, scalable transcription and audio intelligence API to power the Mojo app, with the following specifications:

• Accurate and fast batch transcription for auto-captions at a scalable cost.
• Language support (especially in European languages), to serve the app’s global client base, including the ability to automatically detect not only one’s language but also dialects and accents.
• Top-level precision for word-level timestamps, with the start and end times of each word, detected perfectly, being an essential pre-requisite for video and captions editing.

Solution

Enter Gladia. With Gladia, the Mojo team was able to enhance in-app user experience with the following features:

• Auto-captions, supported in multiple languages, enhanced with a style selection tool.
• Remove pauses, where time-stamped transcripts are used to automatically detect and remove silences from a video.

To enable these features, Gladia relies on an advanced hybrid model architecture with generative AI components. Not only does our API detect words accurately based on acoustic properties of speech, but it also fills the gaps in the transcript based on a contextualized understanding of language, while deploying a range of techniques to eliminate hallucinations and accurately detect a variety of accents even in complex environments.

Impact

By working with the Gladia team to iterate and scale up the volume to thousands of hours transcribed monthly, Mojo saw a noticeable impact on the usage of the auto captions feature, with happy users expressing their satisfaction with the new spot-on quality of the feature.

The Mojo team is continuing to explore the possibilities that transcription brings to the Mojo products and services, and is now considering how they will leverage it in the future with upcoming features like keyword detection.

We're thrilled to be part of this amazing journey with them, and thank Mojo for putting their trust in us! As we move forward, we're excited to team up with more clients, tackle new challenges, and make speech AI more accessible to media companies worldwide.

Having read this case study, do you feel like Gladia can be the right fit for your business too?

Don't hesitate to contact our sales team to discuss this in more detail - or sign up directly below. Beyond we cater to a range of use cases, including virtual meetings, call centers, workspace collaboration, and more.

‍About Gladia

At Gladia, we built an optimized version of Whisper in the form of an API,  adapted to real-life professional use cases and distinguished by exceptional accuracy, speed, extended multilingual capabilities, and state-of-the-art features, including speaker diarization and word-level timestamps. Our latest model, Whisper-Zero, that removes hallucinations and improves accuracy across languages is available now.