Different result with Tensorflow

[tuxzz]

When I measure FLOPs of a Convolution layer with 3x3 kernel, 128 output channels, (1, 1, 128, 128) input data, "VALID" padding, thop and Tensorflow give me different results.

Tensorflow: 38612737
Torch: 20321280.0

Here is my code:

graph = tf.Graph()
with graph.as_default():
  print("* Initialize network")
  v = tf.keras.layers.Conv2D(128, (3, 3), padding="VALID", data_format="channels_first")(tf.placeholder(tf.float32, (1, 1, 128, 128)))
  print("Tensorflow:", tf.profiler.profile(graph, options=tf.profiler.ProfileOptionBuilder.float_operation()).total_float_ops)

print("Torch:",  thop.profile(torch.nn.Conv2d(1, 128, (3, 3)), inputs=(torch.zeros((1, 1, 128, 128)),), verbose=False)[0])

Why?

[tuxzz]

I found the reason.
In thop, the count_convNd has an error that it estimate MAC ops but not FLOPs for convolution operation.

...
  total_ops = y.nelement() * (m.in_channels // m.groups * kernel_ops + bias_ops)
...

In convolution operation, we cut input data into many slices, multiply kernel coefficient with them, then add them into one. So we can modify the code like this:

...
  total_ops = y.nelement() * (m.in_channels // m.groups * kernel_ops * 2 + bias_ops)
...

Result:

Tensorflow: 38612737
Torch: 38610432.0

[Lyken17]

You are right. I will soon push a patch to fix it.

[sytelus]

Can this be merged?

[llbin-oppo]

Is there a mistake here about ConV2D?
bias_ops = 1 if m.bias is not None else 0
I think, bias_ops should be 0 if bias exists, otherwise -1.