Description
openedon Feb 11, 2021
System information
.NET SDK (reflecting any global.json):
Version: 5.0.103
Commit: 72dec52dbdRuntime Environment:
OS Name: Windows
OS Version: 10.0.19042
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\5.0.103\Host (useful for support):
Version: 5.0.3
Commit: c636bbdc8a.NET SDKs installed:
5.0.103 [C:\Program Files\dotnet\sdk].NET runtimes installed:
Microsoft.AspNetCore.All 2.1.25 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.25 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.1.25 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.12 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Issue
After an experiment ended (TimeOut elapsed, with token: CancellationToken.None
)
Sometimes the access of experimentResult.BestRun.Estimator.Fit(_exDataView)
will raise System.OperationCanceledException
.
- What happened?
System.OperationCanceledException: Operation was canceled.
at Microsoft.ML.Runtime.Contracts.CheckAlive(IHostEnvironment env)
at Microsoft.ML.Transforms.ValueToKeyMappingTransformer.Train(IHostEnvironment env, IChannel ch, ColInfo[] infos, IDataView keyData, ColumnOptionsBase[] columns, IDataView trainingData, Boolean autoConvert)
at Microsoft.ML.Transforms.ValueToKeyMappingTransformer..ctor(IHostEnvironment env, IDataView input, ColumnOptionsBase[] columns, IDataView keyData, Boolean autoConvert)
at Microsoft.ML.Transforms.ValueToKeyMappingEstimator.Fit(IDataView input)
at Microsoft.ML.Data.EstimatorChain`1.Fit(IDataView input)
at Microsoft.ML.Data.EstimatorChain`1.Fit(IDataView input)
at Microsoft.ML.Data.EstimatorChain`1.Fit(IDataView input)
at Microsoft.ML.Data.EstimatorChain`1.Fit(IDataView input)
- What did you expect?
This exception does not occur all the time when accessing BestRun
prop.
Source code / logs
var experimentSettings = new MulticlassExperimentSettings
{
MaxExperimentTimeInSeconds = _maxExperimentTimeInSeconds,
CancellationToken = pManager.Token, // CancellationToken.None
OptimizingMetric = MulticlassClassificationMetric.MacroAccuracy
}
var experiment= _mlContext.Auto().CreateMulticlassClassificationExperiment(experimentSettings);
var experimentResult= experiment.Execute(_trainDataView, progressHandler: experimentProgress,
preFeaturizer: _mlContext.Transforms.DropColumns("LabelFeaturized"));
experimentResult.BestRun.Estimator.Fit(_trainDataView);