Skip to content

Need a sample for saving and loading CustomMapping Transform #3117

Closed
@artidoro

Description

@artidoro

We currently only have a simple sample to use the custom mapping transform, but saving and loading is a bit more involved and a sample about that would be very useful.

Here is the current sample that we have:
https://github.com/dotnet/machinelearning/blob/master/docs/samples/Microsoft.ML.Samples/Dynamic/Transforms/CustomMappingSample.cs

Here is a test that shows how to save and load the custom mapping transform:

[CustomMappingFactoryAttribute("MyLambda")]
public class MyLambda : CustomMappingFactory<MyInput, MyOutput>
{
public static void MyAction(MyInput input, MyOutput output)
{
output.Together = $"{input.Float1} + {string.Join(", ", input.Float4)}";
}
public override Action<MyInput, MyOutput> GetMapping()
{
return MyAction;
}
}
[Fact]
public void TestCustomTransformer()
{
string dataPath = GetDataPath("adult.tiny.with-schema.txt");
var source = new MultiFileSource(dataPath);
var loader = ML.Data.CreateTextLoader(new[] {
new TextLoader.Column("Float1", DataKind.Single, 9),
new TextLoader.Column("Float4", DataKind.Single, new[]{new TextLoader.Range(9), new TextLoader.Range(10), new TextLoader.Range(11), new TextLoader.Range(12) })
}, hasHeader: true);
var data = loader.Load(source);
IDataView transformedData;
// We create a temporary environment to instantiate the custom transformer. This is to ensure that we don't need the same
// environment for saving and loading.
var tempoEnv = new MLContext();
var customEst = new CustomMappingEstimator<MyInput, MyOutput>(tempoEnv, MyLambda.MyAction, "MyLambda");
try
{
TestEstimatorCore(customEst, data);
Assert.True(false, "Cannot work without RegisterAssembly");
}
catch (InvalidOperationException ex)
{
if (!ex.IsMarked())
throw;
}
ML.ComponentCatalog.RegisterAssembly(typeof(MyLambda).Assembly);
TestEstimatorCore(customEst, data);
transformedData = customEst.Fit(data).Transform(data);
var inputs = ML.Data.CreateEnumerable<MyInput>(transformedData, true);
var outputs = ML.Data.CreateEnumerable<MyOutput>(transformedData, true);
Assert.True(inputs.Zip(outputs, (x, y) => y.Together == $"{x.Float1} + {string.Join(", ", x.Float4)}").All(x => x));
Done();
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationRelated to documentation of ML.NET

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions