How do you load data from an in-memory data set?

[isaacabraham]

The only sample I can see uses the TextLoader to import data from a CSV file. How do you load in arbitrary data from e.g. a record or array and specify labels etc. on there?

Thanks!

[Ivanidzo4ka]

Sorry, fat fingered issue.
As part of open API, we currently don't have it, but we working on it.
As part of internals we have ArrayDataViewBuilder and you can see example of it usage here:

machinelearning/test/Microsoft.ML.Predictor.Tests/TestTransposer.cs

Line 148 in c023727

ArrayDataViewBuilder builder = new ArrayDataViewBuilder(Env);

[isaacabraham]

Yep, understood. Not expecting everything to be there, but this is definitely something you want to have - the ability to pass in arbitrary data.

[Ivanidzo4ka]

For one who will work on this in future.
we have DataViewConstructionUtils internal class which does desired.

machinelearning/src/Microsoft.ML.Api/DataViewConstructionUtils.cs

Line 17 in c023727

internal static class DataViewConstructionUtils

Maybe we need to add ArrayDataView, and we need to decide should we expose IHostEnvironment or not, and where in new API it belongs.

[Ivanidzo4ka]

also copy of #10

[isaacabraham]

@Ivanidzo4ka whichever way you go, make it as simple as possible to consume. This means either allowing F# Records / C# Classes, with either attributes or some guidance to specify labels / features etc., or (and this should definitely also be supported) support for naked arrays or sequences of either arrays (so a 2x2 array) or tuple data.

[danroot]

Looking for this as well. A couple use case I can see are:

1. Loaders other than flat files. SqlLoader, etc
   2)Scenarios where the schema is not known at compile time - ie a command line tool that lets users pass in arbitrary csv file.

[TomFinley]

Hi @isaacabraham and @danroot , thanks for the feedback. @Ivanidzo4ka introduced an attempt to solve this issue in #106 , though I see that PR did not link to this issue but rather the pre-existing #10 . Is it possible for you to try it, and provide feedback on whether it does what you need?

[isaacabraham]

Will give it a bash in the coming days - thanks!