Tom/tflite dequantize #1322
Tom/tflite dequantize #1322
Conversation
Signed-off-by: Tom Wildenhain <tomwi@microsoft.com>
|
This pull request introduces 2 alerts when merging 3a98be1 into 8114f4b - view on LGTM.com new alerts:
|
Signed-off-by: Tom Wildenhain <tomwi@microsoft.com>
TomWildenhain-Microsoft
force-pushed
the
tom/tflite_dequantize
branch
from
3a98be1 to
f4389b1
Compare
|
NVM disregard this review request until after #1321 merges. It contains changes from both. |
|
This pull request introduces 1 alert when merging 679bc21 into 70bc2b6 - view on LGTM.com new alerts:
|
TomWildenhain-Microsoft
force-pushed
the
tom/tflite_dequantize
branch
from
679bc21 to
394f763
Compare
Signed-off-by: Tom Wildenhain <tomwi@microsoft.com>
TomWildenhain-Microsoft
force-pushed
the
tom/tflite_dequantize
branch
from
394f763 to
c99043a
Compare
| self.structured_outputs = structured_outputs # Needed to determine output order for tf_function | ||
| self.rtol = rtol | ||
| self.atol = atol | ||
| self.ptol = ptol |
There was a problem hiding this comment.
The new models in this commit contain post-processing with an ArgMax op so it makes more sense to allow p percent of the output tensor entries to differ from the TF model rather than allowing for a large atol or rtol. The converted model has a small number of differing entries but they are not necessarily close to to their corresponding tf entries.
| model: "deeplabv3_mnv2_ade20k_uint8.tflite" | ||
| model_type: tflite | ||
| input_get: get_ade20k_uint8 | ||
| ptol: 1.0 |
There was a problem hiding this comment.
This is a 1% error rate which is surprisingly high. Dequantization may mean the ORT result may be slower but actually more accurate than (but different from) the tf results.
Signed-off-by: Tom Wildenhain <tomwi@microsoft.com>
2 participants
No description provided.