FastTree: FastTreeRanking per-iteration loss metrics are empty

[rogancarr]

System information

• OS version/distro: Various Windows 10 machines, Windows Server 2012 or 2016.
• .NET Version (eg., dotnet --info): 4.7, 4.6.x

Although, not a system-related bug.

Issue

• What did you do? Train on a dataset using FastTreeRanking with the testFrequency parameter set to something like 1. Optionally, add a validation set. FastTree will now report training metrics on the train (and validation) sets every N iterations. Do the same for FastTreeRegression and FastTreeClassification.
• What happened? For Regression and Classification, the expected output is printed. For Ranking, The header for the per-iteration test and validation metrics prints, but there are no metrics.
• What did you expect? I expected metrics to be printed for the Ranking case.

Source code / logs

Sample output:

trainNDCG:
validNDCG:
trainNDCG:
validNDCG:
trainNDCG:
validNDCG:
trainNDCG:
validNDCG:

[rogancarr]

I have done a preliminary investigation, and for at least the ranking-sample dataset (which we need to add to this repository), the root cause is that the length of the Dataset.MaxDCG property is 0. Since the length is 0, no scores are computed. It looks like the MaxDCG vector is initialized at double[0][] at the beginning of training and never updated. My working hypothesis is that this is only set during scoring.

Next steps:

• Walk through with a real dataset, verify that the same issue exists