Difference in MOS Scores in Speech Mode between VisQOL 3.3.3 (GitHub) and MATLAB Version with VCTK Dataset

[mahyoseung]

Hello,

I am currently using both VisQOL version 3.3.3 from GitHub and the MATLAB version, and I am facing an issue where the MOS scores differ when using the Speech mode with the VCTK dataset.

Here is the MATLAB code I am using:
% Set the absolute path for the data folder
cleanFolder = 'C:\Data\test_clean';
noiseFolder = 'C:\Data\test_noisy';

% Get a list of all .wav files in the clean and noise folders
cleanFiles = dir(fullfile(cleanFolder, '.wav'));
noiseFiles = dir(fullfile(noiseFolder, '.wav'));

% Ensure the number of files matches
if numel(cleanFiles) ~= numel(noiseFiles)
error('The number of .wav files in the clean and noise folders do not match.');
end

% Initialize an array to store the MOS values
mosValues = zeros(1, numel(cleanFiles));

resultsFile = 'visqol_results.txt';

% Open the results file
fileID = fopen(resultsFile, 'w');

% Evaluate ViSQOL for each file
for i = 1:numel(cleanFiles)
% Load the reference audio file from the clean folder
cleanFile = fullfile(cleanFolder, cleanFiles(i).name);
[reference, fs] = audioread(cleanFile);
reference = mean(reference, 2); % Convert stereo to mono

% Load the degraded audio file from the noise folder
noiseFile = fullfile(noiseFolder, noiseFiles(i).name);
[degraded, ~] = audioread(noiseFile);
degraded = mean(degraded, 2); % Convert stereo to mono

% Evaluate ViSQOL
mos = visqol(degraded, reference, fs, Mode="speech");

% Store the MOS value
mosValues(i) = mos;

% Output the result for each file
fprintf('MOS for file %s: %.2f\n', cleanFiles(i).name, mos);
% Save the result
resultsFolder = 'results';
if ~exist(resultsFolder, 'dir')
    mkdir(resultsFolder);
end
resultsFile = fullfile(resultsFolder, 'visqol_results.csv');
fprintf(fileID, '%s, %.2f\n', cleanFiles(i).name, mos);

end

% Close the results file
fclose(fileID);

% Calculate the average MOS
averageMOS = mean(mosValues);

% Output the average MOS
fprintf('Average MOS for all files: %.2f\n', averageMOS);

have built and am using VisQOL from GitHub based on the provided build instructions.

I am trying to understand why the results differ between the two methods and would appreciate any insights into the cause of the discrepancy.

I have also tried adjusting the window size in MATLAB, but that did not seem to make a difference.

Which method should I trust in this case, and what could be causing these different results between the two versions?

Thank you for your help!