Skip to content

Issue 3234: use model schema type instead of class definition schema #5228

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
frank-dong-ms-zz merged 3 commits into from
Jun 15, 2020
Merged

Issue 3234: use model schema type instead of class definition schema #5228

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
256ec9b
fix issue 3234, use model schema type instead of class definition schema
frank-dong-ms Jun 11, 2020
31cf805
refine comments
frank-dong-ms Jun 11, 2020
4ba050d
change test to test on both case
frank-dong-ms Jun 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions src/Microsoft.ML.Data/DataView/DataViewConstructionUtils.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,13 @@ internal static SchemaDefinition GetSchemaDefinition<TRow>(IHostEnvironment env,
var schemaDefinitionCol = schemaDefinition.FirstOrDefault(c => c.ColumnName == name);
if (schemaDefinitionCol == null)
throw env.Except($"Type should contain a member named {name}");

//Always use column type from model as this type can be more specific.
//This can be corner case:
//For example, we can load an model whose schema contains Vector<Single, 38>
Copy link
Contributor

@mstfbl mstfbl Jun 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will there be a check or at least logging for this corner case? Maybe we should put in a check for arrays of size 0, which is what float[] would be. Also is there a check for the size of provided arrays matching the size of the corresponding Vector?

Copy link
Contributor Author

@frank-dong-ms-zz frank-dong-ms-zz Jun 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test LoadModelWithOptionalColumnTransform is to test this case.
for length check, it is applied at below:
https://github.com/dotnet/machinelearning/blob/master/src/Microsoft.ML.Data/Scorers/SchemaBindablePredictorWrapper.cs#L129-L147

In reply to: 439585282 [](ancestors = 439585282)

//and define this field in input class as float[] without specific array length.
schemaDefinitionCol.ColumnType = col.Type;

var annotations = col.Annotations;
if (annotations != null)
{
Expand Down
18 changes: 17 additions & 1 deletion test/Microsoft.ML.Functional.Tests/ModelFiles.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,7 @@ public class ModelInput
public string[] CategoricalFeatures;
public float[] NumericalFeatures;
#pragma warning restore SA1401
public float Label;
Copy link
Contributor Author

@frank-dong-ms-zz frank-dong-ms-zz Jun 11, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Label [](start = 25, length = 5)

add extra column Label as the model modelwithoptionalcolumntransform's schema contains an extra Label column

}

public class ModelOutput
Expand All @@ -233,14 +234,29 @@ public void LoadModelWithOptionalColumnTransform()
SchemaDefinition inputSchemaDefinition = SchemaDefinition.Create(typeof(ModelInput));
inputSchemaDefinition[nameof(ModelInput.CategoricalFeatures)].ColumnType = new VectorDataViewType(TextDataViewType.Instance, 5);
inputSchemaDefinition[nameof(ModelInput.NumericalFeatures)].ColumnType = new VectorDataViewType(NumberDataViewType.Single, 3);

var mlContext = new MLContext(1);
ITransformer trainedModel;
DataViewSchema dataViewSchema;
trainedModel = mlContext.Model.Load(TestCommon.GetDataPath(DataDir, "backcompat", "modelwithoptionalcolumntransform.zip"), out dataViewSchema);

var modelInput = new ModelInput()
{
CategoricalFeatures = new[] { "ABC", "ABC", "ABC", "ABC", "ABC" },
NumericalFeatures = new float[] { 1, 1, 1 },
Label = 1
};

// test create prediction engine with user defined schema
var model = mlContext.Model.CreatePredictionEngine<ModelInput, ModelOutput>(trainedModel, inputSchemaDefinition: inputSchemaDefinition);
var prediction = model.Predict(new ModelInput() { CategoricalFeatures = new[] { "ABC", "ABC", "ABC", "ABC", "ABC" }, NumericalFeatures = new float [] { 1, 1, 1 } });
var prediction = model.Predict(modelInput);

// test create prediction engine with schema loaded from model
var model2 = mlContext.Model.CreatePredictionEngine<ModelInput, ModelOutput>(trainedModel, inputSchema: dataViewSchema);
var prediction2 = model2.Predict(modelInput);

Assert.Equal(1, prediction.Score[0]);
Assert.Equal(1, prediction2.Score[0]);
}

[Fact]
Expand Down