This is a curated list of awesome Speech Enhancement tutorials, papers, libraries, datasets, tools, scripts and results. The purpose of this repo is to organize the world’s resources for speech enhancement, and make them universally accessible and useful.
To add items to this page, simply send a pull request.
- A literature survey on single channel speech enhancement, 2020 [paper]
- Supervised speech separation based on deep learning: An Overview, 2017 [paper]
- A review on speech enhancement techniques, 2015 [paper]
- Nonlinear speech enhancement: an overview, 2007 [paper]
- Speech enhancement using self-adaptation and multi-head attention, ICASSP 2020 [paper]
- PAN: phoneme-aware network for monaural speech enhancement, ICASSP 2020 [paper]
- Noise tokens: learning neural noise templates for environment-aware speech enhancement [paper]
- Speaker-aware deep denoising autoencoder with embedded speaker identity for speech enhancement, Interspeech 2019 [paper]
- Efficient trainable front-ends for neural speech enhancement, ICASSP 2020 [paper]
- Masking and inpainting: a two-stage speech enhancement approach for low snr and non-stationary noise, ICASSP 2020 [paper]
- A composite dnn architecture for speech enhancement, ICASSP 2020 [paper]
- Multi-domain processing via hybrid denoising networks for speech enhancement, 2018 [paper]
- Speech enhancement using self-adaptation and multi-head attention, ICASSP 2020 [paper]
- Channel-attention dense u-net for multichannel speech enhancement, ICASSP 2020 [paper]
- T-GSA: transformer with gaussian-weighted self-attention for speech enhancement, ICASSP 2020 [paper]
- PAGAN: a phase-adapted generative adversarial networks for speech enhancement, ICASSP 2020 [paper
- Time-frequency masking-based speech enhancement using generative adversarial network, ICASSP 2018 [paper]
- SEGAN: speech enhancement generative adversarial network, Interspeech 2017 [paper]
- Phase reconstruction based on recurrent phase unwrapping with deep neural networks, ICASSP 2020 [paper]
- PAGAN: a phase-adapted generative adversarial networks for speech enhancement, ICASSP 2020 [paper
- Invertible dnn-based nonlinear time-frequency transform for speech enhancement, ICASSP 2020 [paper]
- Phase-aware speech enhancement with deep complex u-net, ICLR 2019 [paper] [code]
- Speech denoising with deep feature losses, Interspeech 2019 [paper]
- End-to-end multi-task denoising for joint sdr and pesq optimization, Arxiv 2019 [paper]
- Improving robustness of deep learning based monaural speech enhancement against processing artifacts, ICASSP 2020 [paper]
| Link | Language | Description |
|---|---|---|
| SETK | Python & C++ | SETK: Speech Enhancement Tools integrated with Kaldi. |
pyAudioAnalysis  |
Python | Python Audio Analysis Library: Feature Extraction, Classification, Segmentation and Applications. |
| Beamformer | Python | Implementation of the mask-based adaptive beamformer (MVDR, GEVD, MCWF). |
| Time-frequency Mask | Python | Computation of the time-frequency mask (PSM, IRM, IBM, IAM, ...) as the neural network training labels. |
| SSL | Python | Implementation of Sound Source Localization. |
| Data format | Python | Format tranform between Kaldi, Numpy and Matlab. |
| Link | Language | Description |
|---|---|---|
| PESQ etc. | Matlab | Evaluation for PESQ, CSIG, CBAK, COVL, STOI |
| SNR, LSD | Python | Evaluation for signal-to-noise-ratio and log-spectral-distortion. |
| SDR | Matlab | Evaluation for signal-to-distortion-ratio. |
| Link | Language | Description |
|---|---|---|
| LPS | Python | Extract log-power-spectrum/magnitude spectrum/log-magnitude spectrum/Cepstral mean and variance normalization. |
MFCC  |
Python | This library provides common speech features for ASR including MFCCs and filterbank energies. |
| pyAudioAnalysis |
Python | Python Audio Analysis Library: Feature Extraction, Classification, Segmentation and Applications. |
| Link | Language | Description |
|---|---|---|
| Data simulation | Python | Add reverberation, noise or mix speaker. |
| RIR simulation | Python | Generation of the room impluse response (RIR) using image method. |
pyroomacoustics  |
Python | Pyroomacoustics is a package for audio signal processing for indoor applications. |
gpuRIR  |
Python | Python library for Room Impulse Response (RIR) simulation with GPU acceleration |
rir_simulator_python  |
Python | Room impulse response simulator using python |
| Name | Utterances | Speakers | Language | Pricing | Additional information |
|---|---|---|---|---|---|
| Dataset by University of Edinburgh (2016) | 35K+ | 86 | English | Free | Noisy speech database for training speech enhancement algorithms and TTS models. |
| TIMIT (1993) | 6K+ | 630 | English | $250.00 | The TIMIT corpus of read speech is one of the earliest speaker recognition datasets. |
| VCTK (2009) | 43K+ | 109 | English | Free | Most were selected from a newspaper plus the Rainbow Passage and an elicitation paragraph intended to identify the speaker's accent. |
| WSJ0 (1993) | -- | 149 | English | $1500 | The WSJ database was generated from a machine-readable corpus of Wall Street Journal news text. |
| LibriSpeech (2015) | 292K | 2K+ | English | Free | Large-scale (1000 hours) corpus of read English speech. |
| CHiME series (~2020) | -- | -- | English | Free | The database is published by CHiME Speech Separation and Recognition Challenge. |
| Name | Noise types | Pricing | Additional information |
|---|---|---|---|
| DEMAND (2013) | 18 | Free | Diverse Environments Multichannel Acoustic Noise Database provides a set of recordings that allow testing of algorithms using real-world noise in a variety of settings. |
| 115 Noise (2015) | 115 | Free | The noise bank for simulate noisy data with clean speech. For N1-N100 noises, they were collected by Guoning Hu and the other 15 home-made noise types by USTC. |
| NoiseX-92 (1996) | 15 | Free | Database of recording of various noises available on 2 CDROMs. |
STOA results in dataset by University of Edinburgh. The following methods are all trained by "trainset_28spk" and tested by common testset. ("F" denotes frequency-domain and "T" is time-domain.)
| Methods | Publish | Domain | PESQ | CSIG | CBAK | COVL | SegSNR | STOI |
|---|---|---|---|---|---|---|---|---|
| Noisy | -- | -- | 1.97 | 3.35 | 2.44 | 2.63 | 1.68 | 0.91 |
| Wiener | -- | -- | 2.22 | 3.23 | 2.68 | 2.67 | 5.07 | -- |
| SEGAN | INTERSPEECH 2017 | T | 2.16 | 3.48 | 2.94 | 2.80 | 7.73 | 0.93 |
| CNN-GAN | APSIPA 2018 | F | 2.34 | 3.55 | 2.95 | 2.92 | -- | 0.93 |
| WaveUnet | arxiv 2018 | T | 2.40 | 3.52 | 3.24 | 2.96 | 9.97 | -- |
| WaveNet | ICASSP 2018 | T | -- | 3.62 | 3.24 | 2.98 | -- | -- |
| U-net | ISMIR 2017 | F | 2.48 | 3.65 | 3.21 | 3.05 | 9.34 | -- |
| MSE-GAN | ICASSP 2018 | F | 2.53 | 3.80 | 3.12 | 3.14 | -- | 0.93 |
| DFL | INTERSPEECH 2019 | T | -- | 3.86 | 3.33 | 3.22 | -- | -- |
| DFL reimplemented | ICLR 2019 | T | 2.51 | 3.79 | 3.27 | 3.14 | 9.86 | -- |
| TasNet | TASLP 2019 | T | 2.57 | 3.80 | 3.29 | 3.18 | 9.65 | -- |
| MDPhD | arxiv 2018 | T&F | 2.70 | 3.85 | 3.39 | 3.27 | 10.22 | -- |
| Complex U-net | INTERSPEECH 2019 | F | 3.24 | 4.34 | 4.10 | 3.81 | 16.85 | -- |
| Complex U-net reimplemented | arxiv 2019 | F | 2.87 | 4.12 | 3.47 | 3.51 | 9.96 | -- |
| SDR-PRSQ | arxiv 2019 | F | 3.01 | 4.09 | 3.54 | 3.55 | 10.44 | |
| RHRnet | ICASSP 2020 | T | 3.20 | 4.37 | 4.02 | 3.82 | 14.71 | 0.98 |
- A Study on WaveNet, GANs and General CNNRNN Architectures, 2019 [link]
- Deep learning: method and applications, 2016 [link]
- Deep learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville, 2016 [link]
- Robust automatic speech recognition by Jinyu Li and Li Deng, 2015 [link]
- CCF speech seminar 2020 [link]
- Real-time Single-channel Speech Enhancement with Recurrent Neural Networks by Microsoft Research, 2019 [link]
- Deep learning in speech by Hongyi Li, 2019 [link]
- High-Accuracy Neural-Network Models for Speech Enhancement, 2017 [link]
- DNN-Based Online Speech Enhancement Using Multitask Learning and Suppression Rule Estimation, 2015 [link]
- Microphone array signal processing: beyond the beamformer,2011 [link]
- Deep learning in speech by Hongyi Li, 2019 [link]
- Learning-based approach to speech enhancement and separation (INTERSPEECH tutorial, 2016) [link]
- Deep learning for speech/language processing (INTERSPEECH tutorial by Li Deng, 2015) [link]
- Speech enhancement algorithms (Stanford University, 2013) [link]