A Fundamental End-to-End Speech Recognition Toolkit and Open Source SOTA Pretrained Models, Supporting Speech Recognition, Voice Activity Detection, Text Post-processing etc.

A PyTorch implementation of the Deep Audio-Visual Speech Recognition paper.

Official Pytorch implementation of "Large Language Models are Strong Audio-Visual Speech Recognition Learners" [ICASSP 2025] and "Mitigating Attention Sinks and Massive Activations in Audio-Visual Speech Recognition with LLMs" [ICASSP 2026].

Official Pytorch implementation of "Omni-AVSR: Towards Unified Multimodal Speech Recognition with Large Language Models" [IEEE ICASSP 2026].

Transformer-based online speech recognition system with TensorFlow 2

Code for InterSpeech 2024 Paper: LipGER: Visually-Conditioned Generative Error Correction for Robust Automatic Speech Recognition

(ICLR 2025) Multi-Task Corrupted Prediction for Learning Robust Audio-Visual Speech Representation

Official source code for the paper "Tailored Design of Audio-Visual Speech Recognition Models using Branchformers"

(SLT 2024) Learning Video Temporal Dynamics with Cross-Modal Attention for Robust Audio-Visual Speech Recognition

End to End Multiview Lip Reading

🤖 📼 Command-line tool for remixing videos with time-coded transcriptions.

Real-Time Audio-visual Speech Recongition

In this repository, I try to use k2, icefall and Lhotse for lip reading. I will modify it for the lip reading task. Many different lip-reading datasets should be added. -_-

🎤 Enable voice recognition for the Doubao input method using Python; ideal for learning and research with a focus on audio processing.

Multi-speaker diarization from video using SyncNet’s cross-modal embedding space to match multiple face tracks to corresponding audio tracks.

🎤 Deploy a simplified voice synthesis service with Fun-CosyVoice3-0.5B-2512, featuring real-time audio output and advanced performance optimizations.