Three major concepts: Estimators, Transformers and Data

[Zruty0]

This is still an incomplete proposal, but I played for a bit with what I had, and it looks promising to me so far.

The general idea is that we narrow our 'zoo' of components (transforms, predictors, scorers, loaders etc) down to three kinds:

• The data. An IDataView with schema, like before.
• The transformer. This is an object that can transform data and output data.

    public interface IDataTransformer
    {
        IDataView Transform(IDataView input);
        ISchema GetOutputSchema(ISchema inputSchema);
    }

• The estimator. This is the 'trainer'. The object that can 'train' a transformer using data.

    public interface IDataEstimator
    {
        IDataTransformer Fit(IDataView input);
        SchemaShape GetOutputSchema(SchemaShape inputSchema);
    }

Obviously, a chain of transformers can itself behave as a transformer, and a chain of estimators can behave like estimators.

We also introduce a 'data reader' (and its estimator), responsible for bringing the data 'from outside' (think loaders):

    public interface IDataReader<TIn>
    {
        IDataView Read(TIn input);
        ISchema GetOutputSchema();
    }

    public interface IDataReaderEstimator<TIn>
    {
        IDataReader<TIn> Fit(TIn input);
        SchemaShape GetOutputSchema();
    }

Old component	New component
Data	Data
Transform	Transformer
Trainable transform (before it is trained)	Estimator
Trainable transform (after it is trained)	Transformer
Trainer	Estimator
Predictor	not sure yet. I'm thinking like 'a field of the scoring transformer?'
Scorer	Transformer
Untrainable loader	Data reader
Trainable loader	Estimator of data reader

I have gone through the motions of creating a 'pipeline estimator' and 'pipeline transformer' objects, which then allows me to write this code to train and test:

            var env = new TlcEnvironment();

            var pipeline = new EstimatorPipe<IMultiStreamSource>(new MyTextLoader(env, MakeTextLoaderArgs()));
            pipeline.Append(new MyConcatTransformer(env, "Features", "SepalLength", "SepalWidth", "PetalLength", "PetalWidth"))
                    .Append(new MyNormalizer(env, "Features"))
                    .Append(new MySdca(env));

            var model = pipeline.Fit(new MultiFileSource(@"e:\data\iris.txt"));

            IrisPrediction[] scoredTrainData = model.Transform(new MultiFileSource(@"e:\data\iris.txt"))
                .AsEnumerable<IrisPrediction>(env, reuseRowObject: false)
                .ToArray();

Here, the only catch is the 'MakeTextLoaderArgs', which is an obnoxiously long way to define the original schema of the text loader. But it is obviously subject to improvement.

The full 'playground' is available at https://github.com/Zruty0/machinelearning/tree/feature/estimators

[Zruty0]

And this is approximately how we can dissect the pipeline (before or after training) and recompose it.

Here, I'm going to strip out a loader and make it into a prediction engine.

            ITransformer<IMultiStreamSource> loader;
            IEnumerable<IDataTransformer> steps;
            (loader, steps) = model.GetParts();

            var engine = new MyPredictionEngine<IrisData, IrisPrediction>(env, loader.GetOutputSchema(), steps);
            IrisPrediction prediction = engine.Predict(new IrisData()
            {
                SepalLength = 5.1f,
                SepalWidth = 3.3f,
                PetalLength = 1.6f,
                PetalWidth = 0.2f,
            });

And this is how I can take out a normalizer, because I'm crazy:

            var bogusEngine = new MyPredictionEngine<IrisData, IrisPrediction>(env, loader.GetOutputSchema(), new[] { steps.First(), steps.Last() });
            IrisPrediction bogusPrediction = bogusEngine.Predict(new IrisData()
            {
                SepalLength = 5.1f,
                SepalWidth = 3.3f,
                PetalLength = 1.6f,
                PetalWidth = 0.2f,
            });

[TomFinley]

Hi @Zruty0 , this seems positive. The "estimator" logic is what we consider the "ideal" solution to #267. It also could be a declarative structure. I would like input from @interesaaat and @tcondie, if they can be persuaded to provide notes. Also separating out the conflation between model and data would avoid #580.

Strong typing seems like a problem in the current proposal. To take an example: a linear trainer produces a linear predictor (ITrainer<LinearPredictor> in the current system). So, if I do var pred = new Sdca(...).Train(...) I can, in a discoverable way through autocomplete, see that this pred has properties for the weights and bias. Or even something like getting the sequence of term mappings out of an instantiated TermTransform, the topics out of LDA, or anything like this. However, this proposal has things are merely estimators that return transforms of no particular type -- opaque black boxes, which is a serious problem people have with the current LearningPipeline API. This would be blocking, but fortunately I'm fairly certain you can resolve this issue with a covariant generic type on the output. (Similar to current arrangement between ITrainer and ITrainer<out TPredictor>, no doubt.)

[alexdegroot]

Regarding MakeTextLoaderArgs:
The options pattern is an option(https://docs.microsoft.com/en-us/aspnet/core/fundamentals/configuration/options?view=aspnetcore-2.1), but you can also consider simply adding an optional 'Action argsAction'-argument into the constructor.
Then you can pretty elegantly configure it:
new MyTextLoader(env, args => { args.ArgOne = 1; args.ArgTwo = "Yada" } );
Or leave it out:
new MyTextLoader(env);

Inside the constructor, you can do such a thing as which ensures that you'll always have an correctly initialized object to deal with for further processing:
var args = argsAction== null ? MakeTextLoaderArgs.Empty : args(new MakeTextLoaderArgs());

[TomFinley]

Oh that's an interesting idea @alexdegroot ... that way you don't have to expose the details of constructing this little object at all. Hmmm. Something about that is very appealing.

[Zruty0]

Yes @alexdegroot , this sounds like a great idea to me.

I am a little bit suspicious of introducing yet another level of indirection into the API (data -> ITransformer that produces data -> IEstimator that produces ITransformer -> Options that are needed to construct IEstimator), but the args-mutating delegate seems easy and powerful.

[alexdegroot]

There's also a chance to do both, simply inject the object as argument or use the arg mutating delegate.
I'm not sure how strict you guys are on single ways of doing stuff.
There's the chance that you end up in the modularity disaster where ASP.NET Core 2 already is: one way works and is preferred in 1.0, another in 2.0 and a third is required to make things work in 2.1.

When it comes to consistency, I'd opt for a single way to produce Args across all these objects. If you want to do things fluent, then basically you should never have to leave your stack of calls. As bonus you can simply comment out a few lines.

[eerhardt]

This might be just a type-o, but what is the difference between ISchema and SchemaShape? I see that estimators operate over SchemaShape and transformers operate on ISchema. Is there a difference? Or are these supposed to be the same thing?