Possible Bug Tensorflow in v0.11 - Invalid argument: input must be 4-dimensional[224,224,3]

[CESARDELATORRE]

Starting from a working ML.NET sample scoring the standard TensorFlow Inception v3 model (it was working on 0.9 and 0.10) after migrating to 0.11 (I just needed to fix a couple of breaking changes in the API) the model is now throwing the following exception:

Exception from TensorFlow:

2019-02-27 14:56:18.800272: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at conv_ops.cc:437 : Invalid argument: input must be 4-dimensional[224,224,3]

EXCEPTION
#########
input must be 4-dimensional[224,224,3]
         [[{{node conv2d0_pre_relu/conv}} = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], padding="SAME", strides=[1, 2, 2, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_input_0_0, conv2d0_w)]]

[CESARDELATORRE]

This other TensorFlow sample using the Estimator for training is having the same issue/exception coming from TensorFlow.

[prathyusha12345]

@zeahmed, @CESARDELATORRE @yaeldekel I am still getting the same error after migrating to Ml.Net v1.0.0-preview.

input must be 4-dimensional[3,224,224]
	 [[{{node conv2d0_pre_relu/conv}} = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], padding="SAME", strides=[1, 2, 2, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_input_0_0, conv2d0_w)]]

The pipeline is below

var pipeline = mlContext.Transforms.LoadImages(outputColumnName: "input", imageFolder: imagesFolder, inputColumnName: nameof(ImageNetData.ImagePath))
                            .Append(mlContext.Transforms.ResizeImages(outputColumnName: "input", imageWidth: ImageNetSettings.imageWidth, imageHeight: ImageNetSettings.imageHeight, inputColumnName: "input"))
                            .Append(mlContext.Transforms.ExtractPixels(outputColumnName: "input"))
                            .Append(mlContext.Model.LoadTensorFlowModel(modelLocation).
                            ScoreTensorFlowModel(outputColumnNames: new[] { "softmax2" },
                                                inputColumnNames: new[] { "input" }));

Could anyone help

[zeahmed]

You need to set addBatchDimensionInput = true in ScoreTensorFlowModel as described here.

machinelearning/src/Microsoft.ML.TensorFlow/TensorFlowModel.cs

Line 57 in 7e0cc46

/// <param name="addBatchDimensionInput">Add a batch dimension to the input e.g. input = [224, 224, 3] => [-1, 224, 224, 3].

[prathyusha12345]

I set this to true. But I am getting the below exception when predicting.

Exception:

input depth must be evenly divisible by filter depth: 224 vs 3
	 [[{{node conv2d0_pre_relu/conv}} = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], padding="SAME", strides=[1, 2, 2, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_input_0_0, conv2d0_w)]]

Code:

var probs = model.Predict(sample).PredictedLabels;

[zeahmed]

My guess is you are passing data as NCHW [3,224,224] while your model expect NHWC.

[[{{node conv2d0_pre_relu/conv}} = Conv2D[T=DT_FLOAT, data_format="NHWC"

see if you can solve it by setting bool interleavePixelColors = true in ExtractPixels

[prathyusha12345]

Yes its working fine after adding the settings interleavePixelColors: true, offsetImage: 117

mlContext.Transforms.ExtractPixels(outputColumnName: "input", interleavePixelColors: ImageNetSettings.channelsLast, offsetImage: ImageNetSettings.mean)

Thanks @zeahmed for helping.

[prathyusha12345]

@zeahmed @yaeldekel I am getting the same exception as shown below in other sample i.e TenosrFlow Estimator sample while predicting using the model here even though I have set up interleavePixelColors = true in ExtractPixels, and adding addBatchDimensionInput: true.

Model Pipeline:

   var pipeline = mlContext.Transforms.Conversion.MapValueToKey(outputColumnName: LabelTokey,inputColumnName:"Label")
                            .Append(mlContext.Transforms.LoadImages(outputColumnName: "input", imageFolder: imagesFolder, inputColumnName: nameof(ImageNetData.ImagePath)))
                            .Append(mlContext.Transforms.ResizeImages(outputColumnName: "input", imageWidth: ImageNetSettings.imageWidth, imageHeight: ImageNetSettings.imageHeight, inputColumnName: "input"))
                            .Append(mlContext.Transforms.ExtractPixels(outputColumnName: "input", interleavePixelColors: ImageNetSettings.channelsLast, offsetImage: ImageNetSettings.mean))
                            .Append(mlContext.Model.LoadTensorFlowModel(featurizerModelLocation).
                                 ScoreTensorFlowModel(outputColumnNames: new[] { "softmax2_pre_activation" }, inputColumnNames: new[] { "input" }, addBatchDimensionInput: true))
                            .Append(mlContext.MulticlassClassification.Trainers.LbfgsMaximumEntropy(labelColumnName:LabelTokey, featureColumnName:"softmax2_pre_activation"))
                            .Append(mlContext.Transforms.Conversion.MapKeyToValue(PredictedLabelValue,"PredictedLabel"))
                            .AppendCacheCheckpoint(mlContext);

Exception:

2019-04-09 11:11:10.709759: W tensorflow/core/framework/op_kernel.cc:1273] OP_REQUIRES failed at conv_ops.cc:437 : Invalid argument: input must be 4-dimensional[224,224,3]

The sample is available here https://github.com/dotnet/machinelearning-samples/tree/migration/1.0.0-preview/samples/csharp/getting-started/DeepLearning_TensorFlowEstimator

Adding @CESARDELATORRE for reference.
Could anyone help