Enable the tuning of WOQ algorithm #1328

Kaihui-intel · 2023-10-17T07:12:28Z

Type of Change

feature

Description

added

WoqTuningParams(Enum)
WeightOnlyQuantSampler
Woq tuning will be at the beginning of the basic strategy or auto strategy

Usage

Currently, it's only available for the ORT backend with approach is weight_only.
It will be triggered when a) quant_level is auto or b) quant_level is 1 && strategy is basic.
algorithm is not set in op_type_dict

# config example
conf = PostTrainingQuantConfig(
    approach="weight_only",
    quant_level="auto",
)

Strategy support woq tuning
ORT backend ready
Docs, The document will be updated in another PR

Expected Behavior & Potential Risk

How has this PR been tested?

Pre-CI

Dependency Change?

None

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

…mpressor into kaihui/wo_tuning

for more information, see https://pre-commit.ci

…mpressor into kaihui/wo_tuning

for more information, see https://pre-commit.ci

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

for more information, see https://pre-commit.ci

neural_compressor/strategy/basic.py

neural_compressor/strategy/utils/tuning_sampler.py

test/strategy/test_basic.py

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

…mpressor into kaihui/wo_tuning

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

for more information, see https://pre-commit.ci

…mpressor into kaihui/wo_tuning

neural_compressor/strategy/strategy.py

neural_compressor/strategy/utils/utility.py

test/strategy/test_onnx.py

Signed-off-by: yuwenzho <yuwen.zhou@intel.com>

for more information, see https://pre-commit.ci

Signed-off-by: yuwenzho <yuwen.zhou@intel.com>

for more information, see https://pre-commit.ci

Signed-off-by: yuwenzho <yuwen.zhou@intel.com>

for more information, see https://pre-commit.ci

neural_compressor/adaptor/onnxrt.py

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

* support WOQ algos tuning --------- Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Signed-off-by: yuwenzho <yuwen.zhou@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: yuwenzho <yuwen.zhou@intel.com> Signed-off-by: bmyrcha <bartosz.myrcha@intel.com>

* support WOQ algos tuning --------- Signed-off-by: Kaihui-intel <kaihui.tang@intel.com> Signed-off-by: yuwenzho <yuwen.zhou@intel.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: yuwenzho <yuwen.zhou@intel.com>

Kaihui-intel added 3 commits October 17, 2023 14:35

support WOQ algo tuning

cb696fb

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

add WoqTuningParams docstring

1bb5d85

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

Merge branch 'kaihui/wo_tuning' of https://github.com/intel/neural-co…

84f4bc4

…mpressor into kaihui/wo_tuning

Kaihui-intel changed the title ~~Support the tuning of WOQ algo in strategy layer~~ Support the tuning of WOQ algorithm in strategy layer Oct 17, 2023

pre-commit-ci bot and others added 5 commits October 17, 2023 07:13

[pre-commit.ci] auto fixes from pre-commit.com hooks

9afdc8c

for more information, see https://pre-commit.ci

Merge branch 'kaihui/wo_tuning' of https://github.com/intel/neural-co…

25df777

…mpressor into kaihui/wo_tuning

[pre-commit.ci] auto fixes from pre-commit.com hooks

e5119ba

for more information, see https://pre-commit.ci

fix docstring

d0b1d4a

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

5f8cda8

for more information, see https://pre-commit.ci

Kaihui-intel requested review from yiliu30 and yuwenzho October 17, 2023 07:34

yiliu30 reviewed Oct 17, 2023

View reviewed changes

neural_compressor/strategy/basic.py Show resolved Hide resolved

neural_compressor/strategy/basic.py Outdated Show resolved Hide resolved

neural_compressor/strategy/utils/tuning_sampler.py Show resolved Hide resolved

test/strategy/test_basic.py Outdated Show resolved Hide resolved

Kaihui-intel and others added 5 commits October 18, 2023 15:54

support woq to auto & add ut

073f2cb

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

Merge branch 'kaihui/wo_tuning' of https://github.com/intel/neural-co…

f9b7d43

…mpressor into kaihui/wo_tuning

add docstring for util

223f9eb

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

1bc7c62

for more information, see https://pre-commit.ci

Merge branch 'kaihui/wo_tuning' of https://github.com/intel/neural-co…

1b8ac4a

…mpressor into kaihui/wo_tuning

yiliu30 reviewed Oct 19, 2023

View reviewed changes

neural_compressor/strategy/strategy.py Outdated Show resolved Hide resolved

yiliu30 reviewed Oct 19, 2023

View reviewed changes

neural_compressor/strategy/strategy.py Outdated Show resolved Hide resolved

neural_compressor/strategy/utils/utility.py Outdated Show resolved Hide resolved

test/strategy/test_onnx.py Outdated Show resolved Hide resolved

test/strategy/test_onnx.py Outdated Show resolved Hide resolved

chensuyue added the new feature label Oct 19, 2023

chensuyue added this to the v2.4 milestone Oct 19, 2023

yuwenzho and others added 4 commits October 19, 2023 10:41

support woq tuning for onnxrt

0aa5321

Signed-off-by: yuwenzho <yuwen.zhou@intel.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

ce2dac0

for more information, see https://pre-commit.ci

update UT of onnxrt woq tuning

d87ada6

Signed-off-by: yuwenzho <yuwen.zhou@intel.com>

Merge branch 'master' into kaihui/wo_tuning

b411551

yuwenzho requested a review from mengniwang95 October 19, 2023 02:49

pre-commit-ci bot and others added 3 commits October 19, 2023 02:49

[pre-commit.ci] auto fixes from pre-commit.com hooks

9e5537f

for more information, see https://pre-commit.ci

update dataloader check for onnxrt woq

1369c7f

Signed-off-by: yuwenzho <yuwen.zhou@intel.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

e95fb8b

for more information, see https://pre-commit.ci

yiliu30 approved these changes Oct 19, 2023

View reviewed changes

mengniwang95 reviewed Oct 19, 2023

View reviewed changes

neural_compressor/adaptor/onnxrt.py Show resolved Hide resolved

mengniwang95 approved these changes Oct 19, 2023

View reviewed changes

yiliu30 changed the title ~~Support the tuning of WOQ algorithm in strategy layer~~ Enable the tuning of WOQ algorithm Oct 19, 2023

yuwenzho and others added 2 commits October 20, 2023 10:06

Merge branch 'master' into kaihui/wo_tuning

8ba8272

remove recipes from woq docstring

b409d1c

Signed-off-by: Kaihui-intel <kaihui.tang@intel.com>

yiliu30 merged commit 6d4ea5b into master Oct 20, 2023

yiliu30 deleted the kaihui/wo_tuning branch October 20, 2023 08:57

Kaihui-intel mentioned this pull request Oct 25, 2023

Update woq tuning document #1347

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Enable the tuning of WOQ algorithm #1328

Enable the tuning of WOQ algorithm #1328

Uh oh!

Kaihui-intel commented Oct 17, 2023 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Enable the tuning of WOQ algorithm #1328

Enable the tuning of WOQ algorithm #1328

Uh oh!

Conversation

Kaihui-intel commented Oct 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Type of Change

Description

Usage

Expected Behavior & Potential Risk

How has this PR been tested?

Dependency Change?

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Kaihui-intel commented Oct 17, 2023 •

edited

Loading