updated pipeline
This commit is contained in:
Plachta
@@ -1,19 +1,23 @@
|
||||
[中文文档请点击这里](https://github.com/Plachtaa/VITS-fast-fine-tuning/blob/main/README_ZH.md)
|
||||
# VITS Fast Fine-tuning
|
||||
This repo will guide you to add your own character voices, or even your own voice, into an existing VITS TTS model
|
||||
This repo will guide you to add your own character voices, or even your own voice, into existing VITS TTS model
|
||||
to make it able to do the following tasks in less than 1 hour:
|
||||
|
||||
1. Any-to-any voice conversion between you & any characters you added & preset characters
|
||||
1. Many-to-many voice conversion between any characters you added & preset characters in the model.
|
||||
2. English, Japanese & Chinese Text-to-Speech synthesis with the characters you added & preset characters
|
||||
|
||||
|
||||
Welcome to play around with the base model, a Trilingual Anime VITS!
|
||||
[](https://huggingface.co/spaces/Plachta/VITS-Umamusume-voice-synthesizer)
|
||||
Welcome to play around with the base models!
|
||||
Chinese & English & Japanese：[](https://huggingface.co/spaces/Plachta/VITS-Umamusume-voice-synthesizer)
|
||||
|
||||
Chinese & Japanese：[](https://huggingface.co/spaces/sayashi/vits-uma-genshin-honkai)
|
||||
|
||||
|
||||
### Currently Supported Tasks:
|
||||
- [x] Convert user's voice to characters listed [here](https://github.com/SongtingLiu/VITS_voice_conversion/blob/main/configs/finetune_speaker.json)
|
||||
- [x] Chinese, English, Japanese TTS with user's voice
|
||||
- [x] Chinese, English, Japanese TTS with custom characters!
|
||||
- [x] Clone character voice from 10+ short audios
|
||||
- [x] Clone character voice from long audio(s) >= 3 minutes (one audio should contain single speaker only)
|
||||
- [x] Clone character voice from videos(s) >= 3 minutes (one video should contain single speaker only)
|
||||
- [x] Clone character voice from BILIBILI video links (one video should contain single speaker only)
|
||||
|
||||
### Currently Supported Characters for TTS & VC:
|
||||
- [x] Umamusume Pretty Derby (Used as base model pretraining)
|
||||
@@ -25,48 +29,22 @@ Welcome to play around with the base model, a Trilingual Anime VITS!
|
||||
|
||||
|
||||
## Fine-tuning
|
||||
It's recommended to perform fine-tuning on [Google Colab](https://colab.research.google.com/drive/1omMhfYKrAAQ7a6zOCsyqpla-wU-QyfZn?usp=sharing)
|
||||
It's recommended to perform fine-tuning on [Google Colab](https://colab.research.google.com/drive/1pn1xnFfdLK63gVXDwV4zCXfVeo8c-I-0?usp=sharing)
|
||||
because the original VITS has some dependencies that are difficult to configure.
|
||||
|
||||
### How long does it take?
|
||||
1. Install dependencies (2 min)
|
||||
2. Record at least 20 your own voice, the content to read will be presented in UI, less than 20 words per sentence. (5~10 min)
|
||||
3. Upload your character voices, which should be a `.zip` file,
|
||||
it's file structure should be like:
|
||||
```
|
||||
Your-zip-file.zip
|
||||
├───Character_name_1
|
||||
├ ├───xxx.wav
|
||||
├ ├───...
|
||||
├ ├───yyy.mp3
|
||||
├ └───zzz.wav
|
||||
├───Character_name_2
|
||||
├ ├───xxx.wav
|
||||
├ ├───...
|
||||
├ ├───yyy.mp3
|
||||
├ └───zzz.wav
|
||||
├───...
|
||||
├
|
||||
└───Character_name_n
|
||||
├───xxx.wav
|
||||
├───...
|
||||
├───yyy.mp3
|
||||
└───zzz.wav
|
||||
```
|
||||
Note that the format & name of the audio files does not matter as long as they are audio files.
|
||||
Audio quality requirements: >=2s, <=10s per audio, background noise should be as less as possible.
|
||||
Audio quantity requirements: at least 10 per character, better if 20+ per character.
|
||||
You can either choose to perform step 2, 3, or both, depending on your needs.
|
||||
1. Install dependencies (3 min)
|
||||
2. Choose pretrained model to start. The detailed differences between them are described in [Colab Notebook](https://colab.research.google.com/drive/1pn1xnFfdLK63gVXDwV4zCXfVeo8c-I-0?usp=sharing)
|
||||
3. Upload the voice samples of the characters you wish to add，see [DATA.MD](https://github.com/Plachtaa/VITS-fast-fine-tuning/blob/main/DATA.MD) for detailed uploading options.
|
||||
4. Start fine-tuning. Time taken varies from 20 minutes ~ 2 hours, depending on the number of voices you uploaded.
|
||||
|
||||
4. Fine-tune (30 min)
|
||||
After everything is done, download the fine-tuned model & model config
|
||||
|
||||
## Inference or Usage (Currently support Windows only)
|
||||
0. Remember to download your fine-tuned model!
|
||||
1. Download the latest release
|
||||
2. Put your model & config file into the folder `inference`, make sure to rename the model to `G_latest.pth` and config file to `finetune_speaker.json`
|
||||
2. Put your model & config file into the folder `inference`, which are named `G_latest.pth` and `finetune_speaker.json`, respectively.
|
||||
3. The file structure should be as follows:
|
||||
```shell
|
||||
```
|
||||
inference
|
||||
├───inference.exe
|
||||
├───...
|
||||
@@ -74,3 +52,7 @@ inference
|
||||
└───G_latest.pth
|
||||
```
|
||||
4. run `inference.exe`, the browser should pop up automatically.
|
||||
|
||||
## Use in MoeGoe
|
||||
0. Prepare downloaded model & config file, which are named `G_latest.pth` and `moegoe_config.json`, respectively.
|
||||
1. Follow [MoeGoe](https://github.com/CjangCjengh/MoeGoe) page instructions to install, configure path, and use.
|
||||
Reference in New Issue
Block a user