Open
Description
openedon Dec 1, 2021
The inference results obtained using onnxruntime and pytorch are different. The code I used is shown below, the model only contains a linear layer.
When the pytorch and onnxruntime running results are the same, the allclose function should output True. However, on one ubuntu16.04 server with 2080ti, the output is True, and on another ubuntu20.04 server with 3090, the output is False.
System information
- OS Platform and Distribution: Ubuntu 16.04 / Ubuntu 20.04
- ONNX Runtime installed from (source or binary): binary / binary
- ONNX Runtime version: onnxruntime-gpu 1.9.0 / onnxruntime-gpu 1.9.0
- Python version: 3.8.3 / 3.8.10
- CUDA/cuDNN version: 11.0 / 11.2
- GPU model and memory: 2080ti / 3090
class LinearTest(nn.Module):
def __init__(self, n_in, n_out, ng=32, act=True):
super().__init__()
self.linear = nn.Linear(n_in, n_out, bias=False)
def forward(self, x):
out = self.linear(x)
return out
test_net = LinearTest(32, 128)
test_net.eval()
onnx_path = 'test_opset11.onnx'
dummy_input = torch.randn(1, 32)
dummy_input_np = dummy_input.numpy()
test_net.eval()
output_gragh = torch.onnx.export(
model=test_net,
args=(
dummy_input
),
f=onnx_path,
input_names=[
'dummy_input',
],
output_names=['dummy_output'],
dynamic_axes={
'dummy_input': {0: 'batch_size'},
'dummy_output': {0: 'batch_size'},
},
verbose=True,
opset_version=11,
)
torch_out = test_net(dummy_input)
sess_options = ort.SessionOptions()
sess_options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_DISABLE_ALL
ort_sess = ort.InferenceSession(onnx_path, sess_options)
ort_out = ort_sess.run(None, {'dummy_input': dummy_input_np})
print(np.allclose(torch_out.detach().numpy(), ort_out, 0, 1e-4))Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment