Incorrect output saturation in int8 QLinearSoftmax

[mcollinswisc]

Describe the issue

QLinearSoftmax with int8 outputs that should otherwise saturate to 127 (int8 max) instead wrap around to -128. The result of fusing QDQ Softmax to QLinearSoftmax then doesn't match.

It looks like there was a copy-paste error from the uint8 kernel and it's clipping to 255 too in the int8 kernel:

onnxruntime/onnxruntime/contrib_ops/cpu/quantization/qlinear_softmax.cc

Line 246 in a91b0b4

const int8_t vy = static_cast<int32_t>(vq) > 255 ? static_cast<int8_t>(255) : static_cast<int8_t>(vq);

https://github.com/microsoft/onnxruntime/blob/main/docs/ContribOperators.md#com.microsoft.QLinearSoftmax

To reproduce

Run the script in this gist:

https://gist.github.com/mcollinswisc/b3c9a2d9cf50b7bf289ced1d6be2c709

It feeds in a QDQ Softmax graph with and without optimization. In the optimization path, it fuses QLinearSoftmax and runs the buggy kernel.

It's expected to print the below, showing the mismatch between the original QDQ graph and the int8 QLinearSoftmax kernel's output:

optimize=False ops: ['DequantizeLinear', 'Softmax', 'QuantizeLinear']
optimize=True  ops: ['QLinearSoftmax']

optimize=False output:
[[ 127 -128 -128 -128]]

optimize=True output:
[[-128 -128 -128 -128]]

(note the mismatch in the first element, it wrapped around to -128)

Urgency

No external urgency

Platform

Linux

OS Version

Resolute

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.27.0

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response