MyHeyGen | 中文
A civilian version video translation tool that provides a full process solution for voice cloning, translation correction, and lip synthesis
HeyGenClone、TTS、Video-retalking、CodeFormer
- finetune效果 【MyHeyGen测试 | 节选霉霉的NYU毕业演讲片段】
- 【6种语言向世界报喜,我的女儿面面出生啦 ! | MyHeyGen 用例】
- 【好家伙一下子学了英语、日语、法语、俄语、韩语5国外语,肾好,肾好! | MyHeyGen效果演示】
- 【张三老师英文普法!英文区的网友有福啦】
- 【MyHeyGen测试|这英的英语倍儿地道!】
【MyHeyGen教程|这样配置应该简单很多吧】 Equivalent to a one click package, no environment required, but with small funds
- Get huggingface Token in config.json
HF_TOKEN
- Apply for fanyiAPPID and APPKey in config.json
TS_APPID
andTS_APPKEY
- Download weights and unzip it in
MyHeyGen
,download checkpoints then put it in MyHeyGen/video-retalking,remenber copy GFPGANv1.4.pth from weights to checkpoints!
git clone https://github.com/AIFSH/MyHeyGen.git
cd MyHeyGen
bash install.sh
or use docker
docker pull registry.cn-beijing.aliyuncs.com/codewithgpu2/aifsh-myheygen:o3U7yjrWg5
python translate.py /root/MyHeyGen/test/src.mp4 'zh-cn' -o /root/MyHeyGen/test/out_zh.mp4
python translate.py src_video_path lang_code -o out_video_opath
## lang_code in ['en', 'es', 'fr', 'de', 'it', 'pt', 'pl', 'tr', 'ru', 'nl', 'cs', 'ar', 'zh-cn', 'ja','hu','ko']
## Corresponding to [English, Spanish, French, German, Italian, Portuguese, Polish, Turkish, Russian, Dutch, Czech, Arabic, Chinese (Simplified), Japanese, Hungarian, Korean]
- 2023.11.7 add TTS_MODEL in config.json to custom model
- 2023.11.8 update TTS for more reality
- 2023.11.9 fix video-retalking oface error
- 2023.11.10 fix librosa version conflict with latest TTS
- 2023.11.19 Huge update !!! add codeformer,h5 vocal split,rewrite audio aligment,voice-only mode
{
"DET_TRESH": 0.3,
"DIST_TRESH": 0.2,
"DB_NAME": "storage.db",
"HF_TOKEN": "", ## token apply form huggingface
"TS_APPID": "", ## Baidu Fanyi
"TS_APPKEY": "", ## Baidu Fanyi
"HUMAN_TRANS": 0, ## 1 human check; 0 auto
"TTS_MODEL":"tts_models/multilingual/multi-dataset/xtts_v2",
"FT_TTS_MODEL": "" ##the finetune model path to enable xtts fineting mode,
"AUDIO_H5": 0, ## 1 enable H5 for vocal split 0 disable
"VOICE_ONLY": 0 ## 1 enable voice_only mode 0 disable
}
Test on GPU A5000 24GB
python xtts_ft.py luoxiang /root/autodl-tmp/xtts_ft/luoxiang/speaker.WAV /root/autodl-tmp/xtts_ft 3 1
# luoxiang ---specker id
# /root/autodl-tmp/xtts_ft/luoxiang/speaker.WAV ---.wav,.mp4, > 30mins
# /root/autodl-tmp/xtts_ft ---workplace of xtts finetune to save model checkpoints,>20GB
# 3 ---batch_size
# 1 ---1 generate dataset, 0 no genrate again