Egyptian and Nigerian English join the Open ASR Leaderboard
Egyptian and Nigerian English join the Open ASR Leaderboard
A few months ago, we added private data to the Open ASR Leaderboard: 11 splits from Appen and DataOceanAI covering scripted and conversational speech from five accents (blog). The community response was great, with many highlighting the importance of held-out sets for trustworthy evaluations. Since then, we added one more private split for Indian-accented English with Voice Arena (blog).
We’re excited to announce a new collaboration with Liva AI to add even more private data with two new accents: Egyptian 🇪🇬 and Nigerian 🇳🇬.
| Dataset | Accent | Duration [h] | Style | Transcription |
|---|---|---|---|---|
| Liva AI Conversational EG | Egyptian | 1.34 | Conversational, spontaneous | Punctuated, cased, with disfluencies. |
| Liva AI Conversational NG | Nigerian | 1.55 | Conversational, spontaneous | Punctuated, cased, with disfluencies. |
This is part of our continued effort to improve representation in ASR evals (more accents and more languages) and to improve trust (new private data to minimize benchmaxxing, and new data providers). We’re excited to see the community continuing to push ASR performance for the world 🤗
Richer accent breakdown in the Private tab
With these new accents, we can now present a richer breakdown of accents, based on Kachru’s Three Circles (kudos to Hume AI for this idea in their leaderboard). The Open ASR Leaderboard will now report:
- US vs. non-US (UK, Australia, Canada, India, Nigeria, Egypt)
- Inner Circle (US, UK, Australia, Canada) vs. Outer & Expanding Circles (India, Nigeria, Egypt)
As more accents join, we’ll split the Outer and Expanding Circles.
About Liva AI
Liva AI collects multimodal interaction data through consumer products used by real people. Because Liva owns both the collection environment and distribution, it can design products around specific model gaps across domains, industries, languages, modalities, and interaction types. This gives it access to natural, high-context behavior that emerges through real product use, rather than artificially elicited or task-based interactions.
收藏了,以后慢慢研究。
支持作者,持续关注中。
讲解得很细致,新手也能看懂。
这篇文章分析得很透彻,收藏了!
有没有更详细的教程,期待后续。
实测过类似工具,作者说的基本属实。
有没有更详细的教程,期待后续。
这个比较实用,已转发给同事。
写得挺用心的,支持一下。
路过
实测过类似工具,作者说的基本属实。
整理得太全面了,省了我不少时间。