Change ensembles trainer to work with ITrainerEstimators instead of ITrainers #3796
Conversation
| var pa = new PredictorAndArgs(new SubComponent("WeightedEnsembleMulticlass", "bp=mlr{t-} nm=20 st=BootstrapSelector{} tp=-"), "WE-Bootstrap"); | ||
| Run_TrainTest(pa, TestDatasets.iris); | ||
| Done(); | ||
| } |
There was a problem hiding this comment.
I guess that these tests were taken from TLC. Did you have to regenerate the results or were they the same?
There was a problem hiding this comment.
Good question. I regenerated all the results, but I went back and compared them to the old baselines, and all but one were identical. The different one was for the CV case of the EnsemblesDefaultTest. The differences in the scores were something like this:
7.057208
-0.569736
-3.265584
-2.8438797
-3.42607021
-3.27301645
4.12974262
3.61101723
3.70798635
0.8509629
(the new scores) vs.:
7.346347
-0.308689833
-3.056211
-2.46506
-3.10131264
-2.99300337
4.336238
3.53593183
4.37107658
0.862161636
Do you think we can attribute these changes to a random seed change? Or should we investigate this further?
In reply to: 290953192 [](ancestors = 290953192)
There was a problem hiding this comment.
codemzs
requested review from
ganik,
codemzs and
wschin
and removed request for
Ivanidzo4ka
…Trainers (dotnet#3796) * Change ensembles trainer to work with ITrainerEstimators instead of ITrainers * Use NumberParseOptions.UseSingle when comparing baselines * Decrease number of digits of precision * Decrease number of digits of precision in some tests * Disable some tests on netcoreapp3.0
ghost
locked as resolved and limited conversation to collaborators
This is needed for using ensembles with non-default values for the sub-trainers.
Fixes #3709 .