Per channel fq2i (#8)

Matthew Brookhart · Josh Fromm · web-flow · commit b43ae3059f6d · 2021-08-31T15:47:22.000-06:00
* WIP support per-channel quantization * more WIP * More WIP * fix issue with per-channel bias_add * Fix fake quantize tests (#4) * Fixed fake quantize issues. * Formatting. * Cleanup unused imports * Fix real int8 tests. * Add Relu * One more little one (#5) * Fixed fake quantize issues. * Formatting. * Cleanup unused imports * Fix real int8 tests. * Fix requantize shape bug. * Non-working Per-channel Dense * Fix legalization for non spatial operators. (#6) * Fix legalization for non spatial operators. * Fix axis checks for end2end functionality. * fix axis normalization fix lint fix lint again * Fix bug in requantize dimension expansion. * Format. Co-authored-by: Josh Fromm <jwfromm@octoml.ai>
diff --git a/src/relay/qnn/op/requantize.cc b/src/relay/qnn/op/requantize.cc
@@ -139,11 +139,13 @@ Expr RequantizeLower(const Expr& input_tensor, const Expr& input_scale,
   auto zero_scalar = MakeConstantScalar(DataType::Int(32), 0);
   if (!IsEqualScalar(input_zero_point, zero_scalar)) {
     // Broadcast input zero point if needed.
+    int rank = static_cast<int>(input_shape.size());
+    int axis = (param->axis < 0) ? ((rank > 0) ? rank + param->axis : 0) : param->axis;
     Expr input_zero_broadcast = ExpandBiasToMatchAxis(Reshape(input_zero_point,
                                                               {
                                                                   -1,
                                                               }),
-                                                      input_shape.size(), {param->axis});
+                                                      rank, {axis});
     tensor = Subtract(tensor, Cast(input_zero_broadcast, DataType::Int(32)));
   }
 

Original file line number	Diff line number	Diff line change
`@@ -139,11 +139,13 @@ Expr RequantizeLower(const Expr& input_tensor, const Expr& input_scale,`
`139`	`139`	`auto zero_scalar = MakeConstantScalar(DataType::Int(32), 0);`
`140`	`140`	`if (!IsEqualScalar(input_zero_point, zero_scalar)) {`
`141`	`141`	`// Broadcast input zero point if needed.`
	`142`	`+ int rank = static_cast<int>(input_shape.size());`
	`143`	`+ int axis = (param->axis < 0) ? ((rank > 0) ? rank + param->axis : 0) : param->axis;`
`142`	`144`	`Expr input_zero_broadcast = ExpandBiasToMatchAxis(Reshape(input_zero_point,`
`143`	`145`	`{`
`144`	`146`	`-1,`
`145`	`147`	`}),`
`146`		`- input_shape.size(), {param->axis});`
	`148`	`+ rank, {axis});`
`147`	`149`	`tensor = Subtract(tensor, Cast(input_zero_broadcast, DataType::Int(32)));`
`148`	`150`	`}`
`149`	`151`