VibeVoice
Open-source voice AI models for speech-to-text and text-to-speech, from Microsoft.
- Stars
- 55.1k
- Forks
- 6.2k
- Licence
- MIT
- Language
- Python
- Last update
- 29 days ago
- Type
- AI model / agent
What it is
VibeVoice is a family of open-source voice AI models from Microsoft that cover both speech recognition (turning audio into text) and speech synthesis (text-to-speech). The speech recognition model, VibeVoice-ASR, is designed to transcribe up to 60 minutes of audio in one pass and label who said what and when. Developers use these models to build transcription or voice features without relying on a closed cloud service.
- Transcribe a long recording with speaker labels and timestamps in a single pass
- Add customized hotwords to improve recognition of names or technical terms
- Run streaming transcription that emits text as audio arrives
- Try real-time text-to-speech with the VibeVoice-Realtime-0.5B model via the linked Colab notebook
Great for
- Developers who want to self-host speech-to-text on long recordings such as meetings or interviews
- Teams that need speaker labels and timestamps in transcripts
- Researchers exploring speech models with published technical reports
- Builders who want to add domain terms to improve recognition of names or jargon
Not ideal for
- People who want a finished app with no setup, since this is a model collection aimed at developers
- Users needing text-to-speech from the TTS model specifically, because the README says the VibeVoice-TTS code was removed from the repository in September 2025
- Low-resource devices without a GPU, unless they use the separate BitNet CPU variant, whose details are limited in this README
How to get it
- 1
Clone and build from source
Clone the repository
git clone https://github.com/microsoft/VibeVoice.git - 2
Run it
- Clone the repository and open the docs folder
- Read docs/vibevoice-asr.md for the speech recognition model setup
- Try the hosted ASR Playground at aka.ms/vibevoice-asr before installing locally
- For real-time speech output, open the Colab notebook linked for VibeVoice-Realtime-0.5B
- 3
Get the most out of it
- Use customized hotwords for domain-specific recordings to improve accuracy
- Try the streaming variant if you need text while audio is still arriving
- Check the supported languages list in docs/vibevoice-asr.md, since the README says over 50 languages
- Use the finetuning code in finetuning-asr/ if you need to adapt the model to your own data
Downloads
Strengths
- Backed by Microsoft with linked technical reports and Hugging Face model weights
- Handles long-form audio in one pass and records who spoke and when
- MIT license and wide community interest
Trade-offs
- The README notes the TTS code was removed after misuse concerns, so the TTS story is partly unclear
- Setup details are not fully visible in this truncated README, and running large models likely needs capable hardware
- Some listed variants and features are new, so stability is uncertain
How it compares
| Project | Stars | Type | Level | Updated | Best for |
|---|---|---|---|---|---|
| VibeVoice (this) | 55.1k | AI model / agent | advanced | 29 days ago | Developers who want to self-host speech-to-text on long recordings such as meetings or interviews |
| whisper | 112k | AI model / agent | intermediate | 2 months ago | Developers who want to add transcription of audio files to a Python project |
| pyannote-audio | 10.8k | Library | intermediate | 12 days ago | Developers who want to add speaker separation to a Python audio workflow |
Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.