AutoML 2.0: how to set sampling key

[torronen]

Is your feature request related to a problem? Please describe.
I am trying to run crossvalidation with a sampling key.

However, .SetDataset(data, fold: 10); only accepts the number of folds. I can create cross-validation set with sampling key using ctx.Data.CrossValidationSplit, but the .SetDataset does not have overload to accept a set of TrainTestData.

How can I set my own IReadOnlyList<TrainTestData> cvdata using the new AutoML API ?

Examples:

AutoMLExperiment experiment = ctx.Auto().CreateExperiment();

IReadOnlyList<TrainTestData> cvdata = ctx.Data.CrossValidationSplit(data, numberOfFolds: 10, samplingKeyColumnName: SamplingKeyColumn); // how to use this in the experiment?

            // Configure experiment
experiment
                .SetPipeline(pipeline)
                .SetBinaryClassificationMetric(BinaryClassificationMetric.Accuracy, labelColumn: columnInference.ColumnInformation.LabelColumnName)
                .SetTrainingTimeInSeconds(60)
                .SetDataset(data, fold: 10); //  how to set sampling key?

[torronen]

Is this enough, and how could I confirm sampling key is being used?
How about if I read it from a binary IDV file, then columnInference is not even used when reading, unless sampling column name is saved to the binary file when serializing.

            columnInference.ColumnInformation.SamplingKeyColumnName = SamplingKeyColumn;
            // Create text loader
            TextLoader loader = ctx.Data.CreateTextLoader(columnInference.TextLoaderOptions);
            // Load data into IDataView
            data = loader.Load(trainingDataFiles.ToArray());

(edit: I am now running a test, but so far it would appear it is not using the sampling key due to a accuracy that seems unrealistically high, I may try again with anohter dataset I have results to compare to)

[torronen]

I believe these 3 small updates will fix my issue, at least for now:

https://github.com/torronen/machinelearning/commit/34a0ca1eb9b2e13cf2742be352182072fdf27252

[michaelgsharp]

@torronen could you submit a pr with those fixes?

@LittleLittleCloud can you take a look?