Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Repository files navigation
=================================================
README
=================================================
This is Group 1's (Brian Finnerty, Wiley Matthews, Dat Tran, and Dade Wood)
project submission. The following code is all used in the process of
generating and using diarization datasets to improve diarization model
evaluation.
=================================================
DATASETS
=================================================
We have three directories containing samples of the data we used in this
project as well as a small sample_dataset we generated.
LibriSpeech_sample: Contains sample LibriSpeech files that have audio of
speakers needed to create Mixtures.
musan_sample: Contains musan noise dataset sample files that have audio
of free-sound noise.
sample_dataset: Contains an example dataset that was generated by our code
and the LibriSpeech and musan datasets.
=================================================
CODE
=================================================
rttm.py:
- Contains helper methods and classes for generating Rich Transcription Time Marked (rttm)
files that hold the labels for diarization.
- Running this file will print out tests for creating rttm input.
mixture.py:
- Contains helper methods and classes for generating audio Mixtures for diarization.
A Mixture is a is an audio file on a single channel that is created from combining
multiple audio files. Our Mixture class also generates rttm label files to correspond
with each mixture created.
- To test this individually, modify the filenames that you want to mix in the run_tests()
method and run the python script.
dataset.py:
- Contains helper methods and classes for generating a diarization dataset of Mixture files.
- Examples of generating a dataset are contained in the main code at the end. Directories
must be changed for your local setup to work. Below are descriptions of how to create a dataset.
Creating a dataset:
1) Create the basic dataset settings -
data_dir = directory where audio files of speakers are stored
noise_dir = directory where audio files of noise are stored
output_dir = directory to output the full dataset to
num_samples = number of files to put in the dataset
file_duration_ms = how long you want each file to be in milliseconds
num_unique_speakers = how many unique speakers you want in your dataset
max_num_speakers = max number of speakers allowed in each sample
proportion_only_m = proportion of the dataset you want to contain only male speakers gauranteed
proportion_only_f = proportion of the dataset you want to contain only female speakers gauranteed
2) Create the list of SegmentSettings for every dataset sample to be generated from. A SegmentSetting
is a class for controlling the randomization variables on a particular segment of a diarization Mixture.
position_ms = starting point of this segment for the Mixture in milliseconds
duration_ms = length of this segment in milliseconds
noise_prob = the probability in (0.0, 1.0) that this segment contains noise
num_speakers_probs = list of probabilities in (0.0, 1.0) that define how many speakers may be in this
segment of the Mixture. The list should be of length max_num_speakers+1 and should
add up to 1.
3) Call DiarizationDataset.create_dataset() and a dataset will be generated and saved to output_dir.
dataset_evaluation.ipynb:
- This is a jupyter notebook used to test Hugging Face's diarization pipeline on our created datasets.
- Hugging Face attempts to download models to a directory that is protected by some computers'
filesystem security. In order to get this to work we utilized google colab to run the notebook.
- Further instructions are located in the notebook.