HomeXiaomi's OmniVoice Open-Source TTS Model Enables Zero-Shot Cloning Across 600+ Languages
Recently, the next-generation Kaldi team (k2-fsa) at Xiaomi officially open-sourced OmniVoice, a massive multilingual zero-shot text-to-speech model that supports over 600 languages. It achieves state-of-the-art results across multiple key benchmarks for Chinese, English, and multilingual synthesis, marking a significant breakthrough in the field.
Leading Performance: Chinese WER as Low as 0.84%, Outperforming Mainstream Models in Multilingual Tests
On the Seed-TTS Chinese test set, OmniVoice achieves a remarkably low word error rate (WER) of just 0.84%. In multilingual evaluations, its similarity (SIM-o) and WER scores surpass well-known commercial models like ElevenLabs v2 and MiniMax, demonstrating exceptional speech naturalness and clarity.
Ultra-Fast Inference: RTF as Low as 0.025, 40x Faster Than Real-Time
OmniVoice boasts a real-time factor (RTF) as low as 0.025, meaning its synthesis speed far exceeds real-time requirements. This massive efficiency gain enables the rapid generation of long-form speech in practical applications, greatly enhancing the user experience.
Core Architectural Innovation: Discrete Non-Autoregressive Design Inspired by Diffusion Models
OmniVoice employs a novel discrete non-autoregressive architecture inspired by diffusion language models. It generates speech from text in a single step, bypassing traditional intermediate semantic tokens. This streamlined design simplifies the pipeline while maintaining high output quality. A full codebook random masking strategy, combined with pre-trained LLM initialization, further boosts training efficiency and improves the final speech's clarity and intelligibility.
Flexible Voice Cloning & Customization: Works with Just 3-10 Seconds of Audio
The model supports high-quality zero-shot voice cloning using only 3-10 seconds of reference audio. Users can also customize voice attributes through natural language prompts, specifying gender, age, pitch, accent, dialect, and even special effects like whispering.
Handles Non-Linguistic Symbols & Fine-Grained Pronunciation Control
OmniVoice can process non-linguistic symbols, such as [laughter], and supports pronunciation correction via pinyin or phonetic symbols. This makes it particularly well-suited for precise synthesis in Chinese and various dialects.
Support for 600+ Languages: Aiding Digital Preservation of Minority and Endangered Languages
A key highlight of OmniVoice is its extensive language coverage, efficiently supporting both major and numerous low-resource languages. For minority and endangered languages, it can generate high-quality speech with minimal data samples, offering significant potential for digital language preservation and cultural protection.
OmniVoice's code and pre-trained models are now open-sourced on GitHub and Hugging Face, enabling developers to deploy it locally or integrate it into applications. AIbase will continue to monitor community feedback and real-world use cases. Developers are encouraged to share their experiences.
Project Link: https://github.com/k2-fsa/OmniVoice
Related article
Stripe to Acquire OpenRouter as AI Model Routing Gains Momentum
Stripe has agreed to acquire OpenRouter, an AI model-routing platform that gives developers access to hundreds of models through a single interface. The deal adds model selection and routing to Stripe’s existing work around AI usage and token-based b
Former OpenAI Researcher Barrett Zoph Returns to Google to Support Gemini Development
Barret Zoph, who previously abruptly left and joined OpenAI due to a disagreement with Thinking Machines Lab CEO Miriam Mularati, has once again changed employers. Alphabet's Google announced that Zoph will return as Vice President of Research, bring
How Recursive Intelligence Raised $335M at $4B Valuation in 4 Months
The co-founders of startup Ricursive Intelligence seemed destined to be co-founders.Anna Goldie, CEO, and Azalia Mirhoseini, CTO, are so well-known in the AI community that they were among those AI engineers who “got those weird emails from Zuckerber
Related Special Topic Recommendations
Design & Art
Top AI Illustration Tools: Create Editorial Scenes and Cover Art Faster
2026 Latest Top-rated AI Illustration Tools: Our curated list features the best powerfully designed tools for creating stunning editorial scenes and eye-catching cover art quickly. Each option has gone through rigorous real-world tests, offering a free vs paid comparison to help you find the perfect match for your workflow. Try these must-try solutions today to unlock your AI edge and boost your creative productivity. Explore now at XIX.AI.
10 tools
xix.ai
Education and Learning
Top AI Tutors for Exam Prep: Practice Math, Science, and English
2026 Latest Top-Rated Best AI Tutors for Exam Prep are here on XIX.AI! Our curated list features powerful game-changing tools that offer free vs paid comparison, real-world tests, and weekly updated rankings. They help boost your writing efficiency, simplify complex topic learning, and unlock your academic potential. Explore now to Discover your perfect exam prep tool and Start creating today to Unlock your AI edge!
10 tools
xix.ai
Finance
AI Budget Planning Tools for Personal Money Management
2026 Latest Best Top-Rated AI Budget Planning Tools for Personal Money Management are here on XIX.AI! This curated collection features powerful, game-changing solutions that help you conduct real-world tests, understand spending habits, and create personalized plans to boost productivity. Enjoy a free vs paid comparison along with detailed rankings. Must-try tools designed to simplify money management and unlock your AI edge. Explore now!
11 tools
xix.ai
Health & Wellness
AI Sleep Tracking Assistants: Improve Recovery with Better Bedtime Habits
2026 Latest Best Top-Rated AI Sleep Tracking Assistants curated for ultimate recovery. This fully updated weekly guide features powerful tools that deliver real-world tests, free vs paid comparison, and detailed rankings to help you find the must-try solution that boosts sleep quality and supports better bedtime habits. Explore now at XIX.AI to Unlock your AI edge.
10 tools
xix.ai
Data Analysis
AI SQL Copilots for Product and Revenue Analytics
2026 Latest Best Top-Rated AI SQL Copilots for Product and Revenue Analytics! XIX.AI has curated a powerful, game-changing collection of must-try tools that undergo real-world tests through weekly updated rankings. These tools help users boost productivity significantly by accelerating data analysis, streamlining complex queries, and generating accurate insights to drive smarter business decisions. Explore now to Discover your perfect tool for unlocking your AI edge!
14 tools
xix.ai
Education and Learning
Best AI Homework Helpers That Teach: Learn Concepts, Not Just Answers
2026 Latest Best Top-Rated AI Homework Helpers That Teach Core Concepts Not Just Answers Powerful Tools for Students Must-Try Curated List Weekly Updated Free vs Paid Comparison Real-World Tests Accurate Rankings to Help You Unlock Your Full Learning Potential Explore now to Discover your perfect tool for smarter studying.
10 tools
xix.ai
Comments (0)
0/500
Recently, the next-generation Kaldi team (k2-fsa) at Xiaomi officially open-sourced OmniVoice, a massive multilingual zero-shot text-to-speech model that supports over 600 languages. It achieves state-of-the-art results across multiple key benchmarks for Chinese, English, and multilingual synthesis, marking a significant breakthrough in the field.
Leading Performance: Chinese WER as Low as 0.84%, Outperforming Mainstream Models in Multilingual Tests
On the Seed-TTS Chinese test set, OmniVoice achieves a remarkably low word error rate (WER) of just 0.84%. In multilingual evaluations, its similarity (SIM-o) and WER scores surpass well-known commercial models like ElevenLabs v2 and MiniMax, demonstrating exceptional speech naturalness and clarity.
Ultra-Fast Inference: RTF as Low as 0.025, 40x Faster Than Real-Time
OmniVoice boasts a real-time factor (RTF) as low as 0.025, meaning its synthesis speed far exceeds real-time requirements. This massive efficiency gain enables the rapid generation of long-form speech in practical applications, greatly enhancing the user experience.
Core Architectural Innovation: Discrete Non-Autoregressive Design Inspired by Diffusion Models
OmniVoice employs a novel discrete non-autoregressive architecture inspired by diffusion language models. It generates speech from text in a single step, bypassing traditional intermediate semantic tokens. This streamlined design simplifies the pipeline while maintaining high output quality. A full codebook random masking strategy, combined with pre-trained LLM initialization, further boosts training efficiency and improves the final speech's clarity and intelligibility.
Flexible Voice Cloning & Customization: Works with Just 3-10 Seconds of Audio
The model supports high-quality zero-shot voice cloning using only 3-10 seconds of reference audio. Users can also customize voice attributes through natural language prompts, specifying gender, age, pitch, accent, dialect, and even special effects like whispering.
Handles Non-Linguistic Symbols & Fine-Grained Pronunciation Control
OmniVoice can process non-linguistic symbols, such as [laughter], and supports pronunciation correction via pinyin or phonetic symbols. This makes it particularly well-suited for precise synthesis in Chinese and various dialects.
Support for 600+ Languages: Aiding Digital Preservation of Minority and Endangered Languages
A key highlight of OmniVoice is its extensive language coverage, efficiently supporting both major and numerous low-resource languages. For minority and endangered languages, it can generate high-quality speech with minimal data samples, offering significant potential for digital language preservation and cultural protection.
OmniVoice's code and pre-trained models are now open-sourced on GitHub and Hugging Face, enabling developers to deploy it locally or integrate it into applications. AIbase will continue to monitor community feedback and real-world use cases. Developers are encouraged to share their experiences.
Project Link: https://github.com/k2-fsa/OmniVoice
Stripe to Acquire OpenRouter as AI Model Routing Gains Momentum
Stripe has agreed to acquire OpenRouter, an AI model-routing platform that gives developers access to hundreds of models through a single interface. The deal adds model selection and routing to Stripe’s existing work around AI usage and token-based b
Former OpenAI Researcher Barrett Zoph Returns to Google to Support Gemini Development
Barret Zoph, who previously abruptly left and joined OpenAI due to a disagreement with Thinking Machines Lab CEO Miriam Mularati, has once again changed employers. Alphabet's Google announced that Zoph will return as Vice President of Research, bring
How Recursive Intelligence Raised $335M at $4B Valuation in 4 Months
The co-founders of startup Ricursive Intelligence seemed destined to be co-founders.Anna Goldie, CEO, and Azalia Mirhoseini, CTO, are so well-known in the AI community that they were among those AI engineers who “got those weird emails from Zuckerber
2026 Latest Top-rated AI Illustration Tools: Our curated list features the best powerfully designed tools for creating stunning editorial scenes and eye-catching cover art quickly. Each option has gone through rigorous real-world tests, offering a free vs paid comparison to help you find the perfect match for your workflow. Try these must-try solutions today to unlock your AI edge and boost your creative productivity. Explore now at XIX.AI.
10 tools
xix.ai
2026 Latest Top-Rated Best AI Tutors for Exam Prep are here on XIX.AI! Our curated list features powerful game-changing tools that offer free vs paid comparison, real-world tests, and weekly updated rankings. They help boost your writing efficiency, simplify complex topic learning, and unlock your academic potential. Explore now to Discover your perfect exam prep tool and Start creating today to Unlock your AI edge!
10 tools
xix.ai
2026 Latest Best Top-Rated AI Budget Planning Tools for Personal Money Management are here on XIX.AI! This curated collection features powerful, game-changing solutions that help you conduct real-world tests, understand spending habits, and create personalized plans to boost productivity. Enjoy a free vs paid comparison along with detailed rankings. Must-try tools designed to simplify money management and unlock your AI edge. Explore now!
11 tools
xix.ai
2026 Latest Best Top-Rated AI Sleep Tracking Assistants curated for ultimate recovery. This fully updated weekly guide features powerful tools that deliver real-world tests, free vs paid comparison, and detailed rankings to help you find the must-try solution that boosts sleep quality and supports better bedtime habits. Explore now at XIX.AI to Unlock your AI edge.
10 tools
xix.ai
2026 Latest Best Top-Rated AI SQL Copilots for Product and Revenue Analytics! XIX.AI has curated a powerful, game-changing collection of must-try tools that undergo real-world tests through weekly updated rankings. These tools help users boost productivity significantly by accelerating data analysis, streamlining complex queries, and generating accurate insights to drive smarter business decisions. Explore now to Discover your perfect tool for unlocking your AI edge!
14 tools
xix.ai
2026 Latest Best Top-Rated AI Homework Helpers That Teach Core Concepts Not Just Answers Powerful Tools for Students Must-Try Curated List Weekly Updated Free vs Paid Comparison Real-World Tests Accurate Rankings to Help You Unlock Your Full Learning Potential Explore now to Discover your perfect tool for smarter studying.
10 tools
xix.ai
