Added MinMax algorithm for OpenVINO backend #1444

l-bat · 2022-12-20T11:47:55Z

Changes

Added OpenVINO backend support for the MinMax algorithm.

Reason for changes

Extension of the set of algorithms for the OpenVINO backend.

Related tickets

98622

Tests

TBD

AlexKoff88 · 2022-12-20T15:21:28Z

nncf/experimental/openvino_native/graph/metatypes/openvino_metatypes.py

@@ -31,18 +31,42 @@ def get_all_aliases(cls) -> List[str]:
        return cls.op_names


+@dataclass
+class OpWeightDef:


I wonder if this is needed at all to keep only one property of operation? If not, maybe it is more reasonable just to keep his inside operation, e.g. to have weight_channel_axis within the operation.

AlexKoff88 · 2022-12-20T15:23:26Z

nncf/experimental/openvino_native/graph/metatypes/openvino_metatypes.py

+    """
+    Contains the information about the weight of the operation.
+
+    :param weight_channel_axis: Axis for weight per-channel quantization, meaning the number of output filters.


I would suggest using the plural term, like weights and activations everywhere because this is what we use to describe the DL model content, not weight and activation of the model.

Class has been deleted

AlexKoff88 · 2022-12-20T15:24:37Z

nncf/experimental/openvino_native/graph/model_transformer.py

@@ -44,14 +45,19 @@ def transform(self, transformation_layout: TransformationLayout) -> ov.Model:
        :param transformations: lisf of the TransformationCommand transformations.
        """
        output_insert_transformations = []
+        quantizer_insert_transformations = []


following the name below _apply_quantizer_insertion_transformations, you should call it like quantizer_insertion_transformations.

AlexKoff88 · 2022-12-20T15:27:54Z

nncf/experimental/openvino_native/graph/nncf_graph_builder.py

+        :param graph: NNCFGraph.
+        """
+        ov_dtype = node.get_element_type().get_type_name()
+        nncf_dtype = GraphConverter.convert_ov_dtype_to_nncf_dtype(ov_dtype)


Suggest renaming convert_ov_dtype_to_nncf_dtype to convert_to_nncf_dtype

AlexKoff88 · 2022-12-20T15:29:53Z

nncf/experimental/openvino_native/graph/nncf_graph_builder.py

+        ov_dtype = node.get_element_type().get_type_name()
+        nncf_dtype = GraphConverter.convert_ov_dtype_to_nncf_dtype(ov_dtype)
+        node_type = node.get_type_name()
+        metatype = OV_OPERATION_METATYPES.get_operator_metatype_by_op_name(node_type)


Some discrepancy between the naming operation and operator.

AlexKoff88 · 2022-12-20T15:40:03Z

nncf/experimental/openvino_native/quantization/quantize.py

+                  model_type: Optional[ModelType] = None,
+                  ignored_scope: Optional[IgnoredScope] = None) -> ov.Model:
+    """
+    Implementation of the `quantize()` method for the OpenVINO backend via the Model API.


Model API is something that exists in OMZ. I would call it like ...for the OpenVINO backend via the OpenVINO Runtime API.

AlexKoff88 · 2022-12-20T15:45:12Z

nncf/experimental/openvino_native/statistics/collectors.py

+from nncf.experimental.openvino_native.statistics.statistics import OVMeanTensorStatistic
+
+
+class OVNNCFCollectorTensorProcessor(NNCFCollectorTensorProcessor):


Did I get right that we use an external statistic collection mechanism now? Any plans for the in-place statistics?

@l-bat, could you file a ticket based on this comment?

ticket: 100927

nncf/experimental/openvino_native/quantization/algorithms/min_max/openvino_backend.py

alexsu52

Cool staff! Please rebase your PR on develop.

nncf/experimental/openvino_native/quantization/quantize.py

nncf/experimental/openvino_native/quantization/quantizer_parameters.py

alexsu52 · 2023-01-10T06:46:35Z

nncf/experimental/openvino_native/statistics/collectors.py

+from nncf.experimental.openvino_native.statistics.statistics import OVMeanTensorStatistic
+
+
+class OVNNCFCollectorTensorProcessor(NNCFCollectorTensorProcessor):


@l-bat, could you file a ticket based on this comment?

nncf/experimental/openvino_native/graph/nncf_graph_builder.py

nncf/experimental/openvino_native/graph/model_transformer.py

nncf/experimental/openvino_native/graph/nncf_graph_builder.py

nncf/experimental/openvino_native/graph/model_transformer.py

nncf/experimental/openvino_native/quantization/quantizer_parameters.py

nncf/experimental/openvino_native/statistics/aggregator.py

nncf/quantization/algorithms/post_training/algorithm.py

nikita-malininn

Looks great 👍

andrey-churkin · 2023-01-18T11:32:38Z

nncf/experimental/openvino_native/quantization/quantizer_parameters.py

+from dataclasses import dataclass
+import numpy as np


Suggested change

from dataclasses import dataclass

import numpy as np

from dataclasses import dataclass

import numpy as np

andrey-churkin · 2023-01-18T11:37:50Z

tests/openvino/native/common.py

@@ -11,15 +11,20 @@
 limitations under the License.
 """
 import os
+import json


Imports should be grouped in the following order:

Standard library imports.

Related third-party imports.

Local application/library-specific imports.

You should put a blank line between each group of imports.

https://github.com/openvinotoolkit/nncf/blob/develop/docs/styleguide/PyGuide.md#22-imports

alexsu52

LGTM. Please, reply on minor comments from my side.

nncf/common/factory.py

nncf/experimental/openvino_native/quantization/algorithms/min_max/openvino_backend.py

l-bat · 2023-01-19T10:04:13Z

run tensorflow pre-commit tests

AlexKoff88 · 2023-01-19T12:14:04Z

nncf/experimental/openvino_native/graph/model_transformer.py

@@ -37,27 +52,40 @@ def __init__(self, model: ov.Model):
        super().__init__(model)
        self._model = model.clone()
        self.name_to_node_mapping = {op.get_friendly_name(): op for op in self._model.get_ops()}
+        self._model_precision = ModelPrecision.FP32


Even after MO we can have mixed precision when some subgraphs can be still in FP32 while most of the model is in FP16. I wonder why precision is so important on the model level.

I found the answer below but still, I am not sure that we can judge based on the type of the first constant.

ticket: 101738

AlexKoff88 · 2023-01-19T12:29:56Z

nncf/experimental/openvino_native/graph/nncf_graph_builder.py

        return nncf_graph
+
+
+class OVWeightedLayerAttributes(BaseLayerAttributes):


The name does not reflect the content. It can be OVConstPortId or so.

I promise I will rename it in the next PR

l-bat requested a review from nikita-malininn December 20, 2022 11:47

github-actions bot added experimental NNCF Common Pull request that updates NNCF Common NNCF OpenVINO Pull requests that updates NNCF OpenVINO labels Dec 20, 2022

l-bat marked this pull request as draft December 20, 2022 11:48

AlexKoff88 reviewed Dec 20, 2022

View reviewed changes

nikita-malininn reviewed Dec 23, 2022

View reviewed changes

nncf/experimental/openvino_native/quantization/algorithms/min_max/openvino_backend.py Show resolved Hide resolved

l-bat force-pushed the lb/quant_ov_native branch 2 times, most recently from 51d3503 to 7a38b99 Compare January 4, 2023 14:18

l-bat mentioned this pull request Jan 9, 2023

Fix NNCFGraph building for OpenVINO backend #1483

Closed

l-bat marked this pull request as ready for review January 9, 2023 16:18

l-bat force-pushed the lb/quant_ov_native branch from 7107431 to d73e13d Compare January 9, 2023 16:32

alexsu52 reviewed Jan 10, 2023

View reviewed changes

andrey-churkin reviewed Jan 10, 2023

View reviewed changes

nncf/experimental/openvino_native/graph/nncf_graph_builder.py Show resolved Hide resolved

nikita-malininn reviewed Jan 11, 2023

View reviewed changes

l-bat force-pushed the lb/quant_ov_native branch from eddd407 to 2393314 Compare January 16, 2023 14:09

nikita-malininn approved these changes Jan 17, 2023

View reviewed changes

l-bat force-pushed the lb/quant_ov_native branch from d6efded to bd20cc9 Compare January 17, 2023 14:36

l-bat requested review from andrey-churkin, AlexKoff88 and alexsu52 January 18, 2023 08:08

andrey-churkin reviewed Jan 18, 2023

View reviewed changes

andrey-churkin approved these changes Jan 18, 2023

View reviewed changes

l-bat added 18 commits January 18, 2023 17:18

Update tests

abf24b5

Fix FQ calculation and add tests

5ce28af

Add AC

879693f

Add graph tests

d04fcb8

Add docstring

4431aee

Fix comments

017200b

Add comparator for OV models

c3a0d5c

Add references

59de220

Minor fixes

f94295c

Broadcast FQ values and support FP16 models

120d92f

Update references

9d666ab

pylint

9f4001b

fix test

b8cb3eb

Update dates

f2a02fd

rebase

8ec6112

fix pylint

34fbe43

Refactoring

fdc7d02

regen stats

9b59a99

l-bat force-pushed the lb/quant_ov_native branch 2 times, most recently from 9a38c1b to 803b9fc Compare January 18, 2023 17:25

Fixed OV NNCF graph builder

40d14d4

l-bat force-pushed the lb/quant_ov_native branch from 803b9fc to 40d14d4 Compare January 18, 2023 18:05

alexsu52 reviewed Jan 19, 2023

View reviewed changes

nncf/common/factory.py Outdated Show resolved Hide resolved

nncf/common/factory.py Outdated Show resolved Hide resolved

nncf/experimental/openvino_native/quantization/algorithms/min_max/openvino_backend.py Outdated Show resolved Hide resolved

l-bat added 2 commits January 19, 2023 10:28

Change Copyright year

c9af10d

fix imports

c4de3fb

alexsu52 approved these changes Jan 19, 2023

View reviewed changes

AlexKoff88 reviewed Jan 19, 2023

View reviewed changes

alexsu52 merged commit 8079233 into openvinotoolkit:develop Jan 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added MinMax algorithm for OpenVINO backend #1444

Added MinMax algorithm for OpenVINO backend #1444

l-bat commented Dec 20, 2022

AlexKoff88 Dec 20, 2022

l-bat Jan 18, 2023

AlexKoff88 Dec 20, 2022

l-bat Jan 18, 2023

AlexKoff88 Dec 20, 2022

l-bat Jan 18, 2023

AlexKoff88 Dec 20, 2022

l-bat Jan 18, 2023

AlexKoff88 Dec 20, 2022

l-bat Jan 18, 2023

AlexKoff88 Dec 20, 2022 •

edited

Loading

AlexKoff88 Dec 20, 2022

alexsu52 Jan 10, 2023

l-bat Jan 11, 2023

alexsu52 left a comment

alexsu52 Jan 10, 2023

nikita-malininn left a comment

andrey-churkin Jan 18, 2023

andrey-churkin Jan 18, 2023

alexsu52 left a comment

l-bat commented Jan 19, 2023

AlexKoff88 Jan 19, 2023

AlexKoff88 Jan 19, 2023

l-bat Jan 19, 2023

AlexKoff88 Jan 19, 2023

l-bat Jan 19, 2023

		from nncf.experimental.openvino_native.statistics.statistics import OVMeanTensorStatistic


		class OVNNCFCollectorTensorProcessor(NNCFCollectorTensorProcessor):

		return nncf_graph


		class OVWeightedLayerAttributes(BaseLayerAttributes):

Added MinMax algorithm for OpenVINO backend #1444

Added MinMax algorithm for OpenVINO backend #1444

Conversation

l-bat commented Dec 20, 2022

Changes

Reason for changes

Related tickets

Tests

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlexKoff88 Dec 20, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexsu52 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nikita-malininn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alexsu52 left a comment

Choose a reason for hiding this comment

l-bat commented Jan 19, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlexKoff88 Dec 20, 2022 •

edited

Loading