Splitting datasets to train and test sets?

[jwood803]

Is there a way to do this with the current release?

I noticed there's a TrainTestSplitter, but there doesn't seem to be any documentation on how to use it and have been unsuccessful when I was messing with it.

Thanks!

[zeahmed]

Do you mean CrossValidation (CV)? or you just want to split data? CV will be available in the next release. You can have a look at the following CrossValidation test for more details.

machinelearning/test/Microsoft.ML.Tests/Scenarios/SentimentPredictionTests.cs

Line 52 in 465b123

public void CrossValidateSentimentModelTest()

[jwood803]

Just to split the data. I believe the samples already have the data split but was curious if there was a programmatic way within ML.NET to do it.

Thanks for sharing about CrossValidation, though! Looking forward to that!

[Ivanidzo4ka]

If you use myget, or bits from fresh github (this is part of v0.2 which we will release in few days) you can use CollectionDataSource on top of enumerable and just apply any LINQ filters on top of your collection to split it into two halves.

Another option is to replicate CrossValidation behavior.
Create your pipeline, add RandomNumberGenerator to it and then apply RowRangeFilter, first step will generate you column with value [0..1) and second step let you filter out by this column so you can randomly split data into two different sets (one set with range filter greater or equal than certain value, other set with range filter less than certain value)

[TomFinley]

In addition to what @Ivanidzo4ka says about replicating what the code does with the generator and filter, there is also the bootstrap sample transform to consider. In particular: you can generate a partition using default settings of that transform (typically would be used as the trainining set), and alternately with the "compliment" setting enabled (which generates the out-of-bag sample, typically used as test or validation dataset). It would depend on whether you wanted sampling with or without replacement, basically.

[jwood803]

I looked through the source and saw that TestTrainSplit.Split was used, so I tried that out and it seems to split the data nicely.

var env = new TlcEnvironment(42);
var inputFile = new SimpleFileHandle(env, dataPath, false, false);

var dataView = ImportTextData.TextLoader(env, new ImportTextData.LoaderInput()
{
  Arguments =
  {
      HasHeader = true,
      Separator = ","
  },
  InputFile = inputFile
});

var data = TrainTestSplit.Split(env, new TrainTestSplit.Input { Data = dataView.Data, Fraction = 0.8f });

Is this valid to use in applications? I also don't think this can be added to a pipeline, though.

[zeahmed]

Yes, you can use it this way as well. Currently, you won't be able to use it in the pipeline. However, with major revamp in API #371, this will be possible too.

[svickers]

@jwood803 Thanks for the code sample, what do you do with the IDataView once you have it? Having trouble figuring out how to use it with my pipeline like the TextLoader.

[jwood803]

Hey, @svickers!

I'm sure the other folks here can do a better job than I can, but I don't believe the current LearningPipeline API can handle the IDataView. Once the new API comes out, though, you should be able to do a lot with it.

[Ivanidzo4ka]

DRI RESPONSE:
This is an old thread about now legacy API.
How to split data into train test in new API you can find from cookbook:
https://github.com/dotnet/machinelearning/blob/master/docs/code/MlNetCookBook.md#how-do-i-train-using-cross-validation
I'm intent to close this issue within few next day if no one reply.