Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] TFLite quantized ResNet50 test output mismatch when run on arm_cpu target #16534

Open
lhutton1 opened this issue Feb 7, 2024 · 0 comments
Labels
needs-triage PRs or issues that need to be investigated by maintainers to find the right assignees to address it relay:qnn src/relay/qnn topi python/tvm/topi type: bug

Comments

@lhutton1
Copy link
Contributor

lhutton1 commented Feb 7, 2024

Expected behaviour:

No output mismatch.

Actual behaviour:

When compiled on an arm_cpu target, the model results in an output mismatch:

E       AssertionError:
E       Not equal to tolerance rtol=1e-07, atol=1e-07
E
E       Mismatched elements: 3 / 3 (100%)
E       Max absolute difference: 878
E       Max relative difference: 8.69306931
E        x: array([979, 972, 970])
E        y: array([101, 386, 385])

Environment:

Tested with TVM at 6a3fadc. The issue was found as a result of the changes in #16513, however it can be reproduced without as described below.

Run on a Graviton 3 instance.

How to reproduce:

Run the test pytest tests/python/frontend/tflite/test_forward.py -k test_forward_tflite2_qnn_resnet50
with an arm_cpu target. Note: Reminder to remove any skip condition that exists in the test currently.

@lhutton1 lhutton1 added type: bug needs-triage PRs or issues that need to be investigated by maintainers to find the right assignees to address it topi python/tvm/topi frontend:tflite python/tvm/relay/frontend/tflite* relay:qnn src/relay/qnn and removed frontend:tflite python/tvm/relay/frontend/tflite* labels Feb 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-triage PRs or issues that need to be investigated by maintainers to find the right assignees to address it relay:qnn src/relay/qnn topi python/tvm/topi type: bug
Projects
None yet
Development

No branches or pull requests

1 participant