How to define the algorithm of L2_Pool2d?

[mingmingtasd]

As you know, the algorithm of L2_Pool2d is based on the
Lp-normalization function which should be Y = (X1^P + X2^P + ... + Xn^P) ^ (1/P).

But for L2_pool2d, I am not sure whether need to average the sum of elements as
Y =( (X1^2 + X2^2 + ... + Xn^2)/2) ^ (1/2) or directly use Lp-normalization function as
Y = (X1^2 + X2^2 + ... + Xn^2) ^ (1/2).

I find two papers: https://sci-hub.yncjkj.com/10.1109/cvpr.2011.5995370 and https://sci-hub.yncjkj.com/10.1109/tcsvt.2015.2461978, they describe the Lp-normalization as below:

So I confirm that Lp-normalization function should be Y = (X1^P + X2^P + ... + Xn^P) ^ (1/P), but I am still not sure whether need to averge the sum of elements for L2_pool2d. I go through some framwork API spec and find the description as below:

1. NNAPI ANEURALNETWORKS_L2_POOL_2D:
output[b, i, j, c] =
sqrt(sum_{di, dj} pow(input[b, strides[1] * i + di, strides[2] * j + dj, c], 2) /
sum(1))
2. ONNX LpPool:
LpPool consumes an input tensor X and applies Lp pooling across the tensor according to kernel sizes, stride sizes, and pad lengths. Lp pooling consisting of computing the Lp norm on all values of a subset of the input tensor according to the kernel size and downsampling the data into the output tensor Y for further processing.
3. OpenVINO: Not Supported
4. DML DML_LP_POOLING_OPERATOR_DESC :
Computes the Lp-normalized value across the elements within the sliding window over the input tensor. The value of the P variable in the Lp-normalization function Y = (X1^P + X2^P + ... + Xn^P) ^ (1/P), where X1 to Xn representing each of the values within the sliding window. In common use cases, this value is either set to 1 or 2, representing either the L1 or L2 normalization respectively.

So it seems that NNAPI ANEURALNETWORKS_L2_POOL_2D should do averge. But after verifying on DML, DML DML_LP_POOLING_OPERATOR_DESC doesn't averge. Thus the algroithm and implementation for l2_pool2d in these frameworks may be different.

[anssiko]

@huningxin, please report your proposed approach in this issue for the WG to review when you've discussed this issue with @mingmingtasd.

[fdwr]

...I am not sure whether need to average the sum of elements...

@mingmingtasd : What sources are you seeing that average elements before the ^ (1/p)? Everything above appears consistent, including the papers, ONNX, DML, and NNAPI, all supporting Lebesgue Pooling for p = 2 as Y = (X1^2 + X2^2 + ... + Xn^2) ^ (1/2). The / sum(1) in NNAPI is odd but ignorable, as it doesn't affect the result.

[mingmingtasd]

...I am not sure whether need to average the sum of elements...

@mingmingtasd : What sources are you seeing that average elements before the ^ (1/p)? Everything above appears consistent, including the papers, ONNX, DML, and NNAPI, all supporting Lebesgue Pooling for p = 2 as Y = (X1^2 + X2^2 + ... + Xn^2) ^ (1/2). The / sum(1) in NNAPI is odd but ignorable.

Agree. The L2Pool2d should follow and be based on the Lp-normalization function(Y = (X1^P + X2^P + ... + Xn^P) ^ (1/P)).

[fdwr]

@mingmingtasd Is there anything remaining unresolved on this one, or can it be closed?

[mingmingtasd]

@mingmingtasd Is there anything remaining unresolved on this one, or can it be closed?

Let's close it, thanks.

[fujunwei]

TFLite also need to average the count of sum elements ((X1^2 + X2^2 + ... + Xn^2)/n) ^ (1/2), here is the l2_pool2d's kernel, so how to keep compatibility between TFLite models and ONNX models?