Misleading sentence. ML.NET 1.0 allows to do transfer learning on just the final layer of the DNN, not on multiple layers

[CESARDELATORRE]

At the beginning of the tutorial it says:
"What if you could reuse a model that's already been pre-trained to solve a similar problem and retrain either all or some of the layers of that model to make it solve your problem?"

That's a misleading sentence. ML.NET 1.0 allows to do transfer learning on just the final layer on the not on multiple layers of the DNN.

AB#1530397

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 7ca0a402-fa67-fbd4-4cbc-604b4b59c97c
• Version Independent ID: c8a996eb-82ed-a4bd-94d8-ef0b32158c67
• Content: Tutorial: Build an ML.NET custom image classifier with TensorFlow - ML.NET
• Content Source: docs/machine-learning/tutorials/image-classification.md
• Product: dotnet-ml
• GitHub Login: @JRAlexander
• Microsoft Alias: johalex

[JRAlexander]

Hi @CESARDELATORRE, Great feedback! That sentence refers to the following paragraph which explains it further: "Transfer learning includes a few strategies, such as retrain all layers and penultimate layer. This tutorial will explain and show how to use the penultimate layer strategy. The penultimate layer strategy reuses a model that's already been pre-trained to solve a specific problem. The strategy then retrains the final layer of that model to make it solve a new problem. Reusing the pre-trained model as part of your new model will save significant time and resources."

I'd be open to any suggestions you have for clarification. Thanks!

[JRAlexander]

Closing as I believe it's resolved. Please reopen on comment if that isn't the case. Thanks!

[codecolony]

Hello,

I've a very basic question here. If I have a TF model already, do I always need to to transfer learning in order to use it in ML.net? All the examples here point to that way of model consumption. Can you point me to an example where I can do object detection using MobileNet TF model with just loading the model and doing inference with an example image?

[CESARDELATORRE]

@codecolony - No, of course you don't need Transfer Learning and training if you just want to use a TF model (score/inference).

You can load any ONNX or TensorFlow model in ML.NET and use it (score, inference with it).
But it is not as simple as when loading an ML.NET .zip file model since you need to provide additional parameters needed by the DNN.

See TensorFlow examples here, just for scoring/inference:
https://github.com/dotnet/machinelearning-samples/tree/master/samples/csharp/getting-started/DeepLearning_ImageClassification_TensorFlow

https://github.com/dotnet/machinelearning-samples/tree/master/samples/csharp/end-to-end-apps/DeepLearning_ImageClassification_TensorFlow

See an ONNX example here, just for scoring/inference:
https://github.com/dotnet/machinelearning-samples/tree/master/samples/csharp/end-to-end-apps/ObjectDetection-Onnx

[codecolony]

Hi @CESARDELATORRE !,

Thanks a lot for quick response and sharing all the relevant info.
I've been through similar examples earlier but I still have the following confusion. Would be great if you can help me understand a few concepts related to the interfaces working with ML.Net.

1. Why do we have pipeline.fit(data) statement in the above links? It's forcing me to fit data with labels and in my case I just want to input an image with no labels and get output from neural network. Is that possible without using pipeline.fit() statement?

2. My model is SSD MobileNet V2 for object detection (and not just classification) and if I have to prepare data like in the above examples then I have to provide image paths and multiple labels for each image like classes and list of box coordinates. I can understand simple classification example but how do I get my data prepared in this case? This is why I don't want to be concerned with labels here but just take image and then receive classes and boxes as output. Is this possible?

3. Finally, my model provides 3 different outputs - score, box coordinates and class. How can I receive multiple outputs in the code? The following code seems correct? Assume that the tensor names are correct.

.ScoreTensorFlowModel(outputColumnNames: new[] { "scores", "classes", "boxes" }, inputColumnNames: new[] { "input" },
            addBatchDimensionInput:true));

Thank a lot in advance.

[CESARDELATORRE]

For #1:
ML.NET is always based on a pipeline in order to create a model.
In this case you are not fitting data, you call Fit() simply to be able to create a "wrapper model" around the TF or ONNX model. That's why you only need to call Fit() once per application, to create kind of a singleton object instead of doing that "per request or per prediction".

Take a look to the ASP.NET Core app that is using the TF or ONNX models where you'll see it is only called once so it created the wrapper model (ML.NET model) around the TF or ONNX model, but you don't need (and must not do it) to do it on every call.

For #2:
The ONNX sample we have is for Object Detection, you can see a similar case where we are using bounding boxes, etc, and how they are passed to the model.

#3
About this I'm adding @yaeldekel from the team, she might be able to answer better than me. :)