Result of Pad evaluated by onnxruntime is inconsistent with onnx ReferenceEvaluator and onnx doctumentation

[vincentme]

Describe the issue

In mode 'reflect', the result of Pad from onnxruntime is inconsistent with onnx.reference.ReferenceEvaluator, onnx documentation, and numpy.pad.

The test model was create by onnxscript. From the model proto, the model itself should be right. Of course, the evaluated result from onnxscript is also inconsistent. So I created an issue there.

result

numpy:
[[1.  1.2 1.  1.2]
 [2.3 3.4 2.3 3.4]
 [4.5 5.7 4.5 5.7]]

onnxscript evaluator:
Tensor(array([[0. , 1.2, 1. , 1.2],
       [0. , 3.4, 2.3, 3.4],
       [0. , 5.7, 4.5, 5.7]], dtype=float32))

onnxscript evaluator (called onnxruntime?):
[[0.  1.2 1.  1.2]
 [0.  3.4 2.3 3.4]
 [0.  5.7 4.5 5.7]]

onnx ReferenceEvaluator:
[[1.  1.2 1.  1.2]
 [2.3 3.4 2.3 3.4]
 [4.5 5.7 4.5 5.7]]

onnxruntime:
[[0.  1.2 1.  1.2]
 [0.  3.4 2.3 3.4]
 [0.  5.7 4.5 5.7]]

<
   ir_version: 9,
   opset_import: ["" : 20]
>
test (float[?,?] data) => (float[?,?] return_val) {
   const = Constant <value: tensor = int64[4] const {0,2,0,0}> ()
   return_val = Pad <mode: string = "reflect"> (data, const)
}

To reproduce

import numpy as np
from onnxscript import script, FLOAT, opset20 as op
import onnx

data = np.array([
    [1.0, 1.2],
    [2.3, 3.4],
    [4.5, 5.7],
], dtype = np.float32)

mode = 'reflect'

res_np = np.pad(data, ((0, 0), (2, 0)), mode = mode)
res_onnxscript1 = op.Pad(data, pads = [0, 2, 0, 0], mode = mode)

@script()
def test(data: FLOAT[None, None]) -> FLOAT[None, None]:
    return op.Pad(data, pads = [0, 2, 0, 0], mode = mode)
res_onnxscript2 = test(data)

model = test.to_model_proto()
feeds = {'data': data,}

from onnx.reference import ReferenceEvaluator
sess = ReferenceEvaluator(model)
outpus_onnx = sess.run(None, feeds)

from onnxruntime import InferenceSession
sess = InferenceSession(model.SerializeToString())
outpus_onnxruntime = sess.run(None, feeds)

print(f'numpy:\n{res_np}\n')
print(f'onnxscript evaluator:\n{res_onnxscript1}\n')
print(f'onnxscript evaluator (called onnxruntime?):\n{res_onnxscript2}\n')
print(f'onnx ReferenceEvaluator:\n{outpus_onnx[0]}\n')
print(f'onnxruntime:\n{outpus_onnxruntime[0]}\n')

print(onnx.printer.to_text(model))

Urgency

No response

Platform

Linux

OS Version

ubuntu 24.04

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.17.1

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

[github-actions]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

[snnn]

This issue has been automatically closed as 'not planned' because it has been marked as 'stale' for more than 30 days without activity. If you believe this is still an issue, please feel free to reopen it.