Skip to content
Get packs

Guide · Open source

How to use Amphion: clone, build and run audio generation models

Updated · written with AI from the project's own documentation and checked against our catalogue data

Amphion is an open-source Python toolkit for audio, music, and speech generation. It is built for reproducible research and for people learning how these systems work. This guide explains what Amphion contains, how to get the code onto your machine, and who will get the most from it.

Amphionopen-mmlab/Amphion · ★ 10.3k · MIT

Quick start

  1. 1Check prerequisites. Confirm that Python and Git are installed on your machine. Set up a deep-learning environment, since models run locally.
  2. 2Clone the repository. Run the git clone command for open-mmlab/Amphion from the README to download the code.
  3. 3Read the task guide. Open the README for the task you want, such as TTS, voice conversion, or vocoders, and follow its instructions.
  4. 4Build the environment. Follow the repository's build instructions for the components you need, since the README gives only the clone step here.
  5. 5Run a model or metric. Use the scripts and model pages linked from the task guide to run generation or evaluation on your own inputs.

What is Amphion and who is it for?

Amphion is a toolkit for audio, music, and speech generation. Its stated purpose is to support reproducible research and to help junior researchers and engineers get started in the field. It also offers visualizations of classic model architectures to help readers understand how a model works.

It is a research toolkit rather than a finished end-user app. The README describes it that way, so expect to work with code and models yourself.

  • Researchers who want reproducible implementations of speech and audio generation models
  • Junior engineers learning how text-to-speech or voice conversion systems are built
  • Teams experimenting with zero-shot voice cloning or voice conversion
  • People who need vocoders or evaluation metrics for audio generation work

Which tasks does Amphion support?

Amphion lists several generation tasks. The README marks these as supported: text to speech (TTS), singing voice synthesis (SVS), voice conversion (VC), accent conversion (AC), singing voice conversion (SVC), and text to audio (TTA). Text to music (TTM) is marked as in development.

Amphion also includes several vocoders, which turn model output into audio signals, and evaluation metrics for judging generation quality. It bundles models such as MaskGCT and Vevo, and the project mentions large-scale datasets for speech synthesis.

  • TTS: Text to Speech (supported)
  • SVS: Singing Voice Synthesis (supported)
  • VC: Voice Conversion (supported)
  • AC: Accent Conversion (supported)
  • SVC: Singing Voice Conversion (supported)
  • TTA: Text to Audio (supported)
  • TTM: Text to Music (in development)

What do you need before installing Amphion?

The README lists Python and Git as prerequisites. Because the project requires running code and models locally, you also need a working Python and deep-learning environment. Without one, Amphion will not be usable.

The project is licensed under MIT and is written in Python.

  • Python
  • Git
  • A local Python and deep-learning environment, since code and models run on your machine

How do you install Amphion?

Amphion uses a clone-and-build install method. Start by cloning the repository from GitHub with Git. The README gives this command for the clone step. After cloning, follow the instructions in the repository's own README and task folders for building and running each component.

The README gives only the clone command in the material provided here, so check the project's documentation for any further setup steps.

git clone https://github.com/open-mmlab/Amphion.git

Where do you find task-specific documentation?

The main README links to separate guides for each task area. These include guides for TTS, voice conversion, accent conversion, SVC, text to audio, vocoders, and evaluation metrics. Read the guide for the task you care about before running anything.

Several models also have their own pages, such as MaskGCT, Vevo, Vevo1.5, Vevo2, DualCodec, and Metis. Each release note in the README links to its model page or paper.

  • Text-to-speech guide: egs/tts/README.md
  • Voice conversion guide: models/vc/vevo/README.md
  • Singing voice conversion guide: egs/svc/README.md
  • Text-to-audio guide: egs/tta/README.md
  • Vocoder guide: egs/vocoder/README.md
  • Evaluation metrics guide: egs/metrics/README.md

What are the limits to know about?

Amphion is not a turnkey voice app. If you only want to generate speech without any setup, a finished product will suit you better. The latest release is labeled v0.1.1-alpha, so do not assume production-grade stability.

Most of the value is in reading and reproducing research code, so plan time for environment setup and for learning the model pipelines.

  • Not suited to users who want a finished voice app with no setup
  • Not suited to users without a Python and deep-learning environment
  • Not suited to production projects that need guarantees, given the alpha label

What people use it for

  • Reproducing speech generation results from published papers for research
  • Studying how a TTS or voice conversion pipeline is organised, using the architecture visualizations
  • Experimenting with zero-shot voice cloning or voice conversion with bundled models
  • Computing evaluation metrics for audio generation experiments
  • Using vocoders in a custom audio generation pipeline

Frequently asked questions

Is Amphion free to use?

Amphion is released under the MIT licence, as stated in the README. Check the LICENSE file in the repository for the full terms, and check the licence of any dataset or model you use separately.

What do I need to install Amphion?

The README lists Python and Git as prerequisites. You install it by cloning the repository with git clone, then following the build instructions for the parts you need. You also need a local deep-learning environment because models run on your machine.

Is Amphion a text-to-speech app I can use right away?

No. The README describes Amphion as a research toolkit rather than an end-user product. It suits people comfortable running code and models locally.

What is Amphion's latest version?

The README labels the latest release as v0.1.1-alpha. Treat it as alpha software and avoid relying on it for production work without testing.

How does Amphion compare to Coqui TTS, Bark or AudioCraft?

Amphion covers several tasks in one codebase, including TTS, voice conversion, singing voice, text to audio, vocoders and metrics. The comparable projects listed here are ESPnet, Coqui TTS, Bark and AudioCraft. Compare them on the tasks, models and licences you need, since this guide does not benchmark them.

Does Amphion support voice cloning?

The README says Amphion bundles zero-shot models such as Vevo and MaskGCT, and lists voice conversion and TTS as supported tasks. Check each model page for how cloning works and what inputs it needs.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

Amphion suits researchers, students and engineers who want reproducible speech and audio generation code they can read and run locally. It is a poor fit for anyone who wants a finished voice app with no setup or who needs production-grade stability from an alpha release.

Install options & downloads