This repository was archived by the owner on Sep 18, 2024. It is now read-only.
[Compression] Add quantization tutorial #5454
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
matluster
reviewed
Mar 31, 2023
dependencies/recommended.txt
Outdated
| @@ -21,3 +21,4 @@ matplotlib | |||
| git+https://github.com/microsoft/nn-Meter.git#egg=nn_meter | |||
| sympy | |||
| timm >= 0.5.4 | |||
| datasets == 2.10.1 | |||
There was a problem hiding this comment.
Any particular reason to freeze the version?
There was a problem hiding this comment.
good suggestion, there is no need to freeze the version. I will modify it.
J-shang
reviewed
Apr 3, 2023
J-shang
reviewed
Apr 17, 2023
docs/source/examples.rst
Outdated
|
|
||
| .. cardlinkitem:: | ||
| :header: Quantize Bert on Task MNLI | ||
| :description: An end to end example for how to using NNI to quantize transformer and show the real speedup number |
J-shang
reviewed
Apr 17, 2023
| Quantize BERT on Task GLUE | ||
| ========================== | ||
|
|
||
| Here we show an effective transformer simulated quantization process that NNI team has tried, and users can use NNI to discover better process |
J-shang
reviewed
Apr 17, 2023
|
|
||
| Here we show an effective transformer simulated quantization process that NNI team has tried, and users can use NNI to discover better process | ||
|
|
||
| we use the BERT model and the trainer pipeline in the Transformers to do some experiments. |
J-shang
reviewed
Apr 17, 2023
| # .. note:: | ||
|
|
||
| # Please set ``is_trace`` to ``False`` to fine-tune the BERT model and set ``is_trace`` to ``True`` | ||
| # When you need to create a traced trainer for model quantization. |
There was a problem hiding this comment.
and fine-tuning model can also use a traced trainer.
J-shang
reviewed
Apr 17, 2023
| config_list = [{ | ||
| 'op_types': ['Linear'], | ||
| 'op_names_re': ['bert.encoder.layer.{}'.format(i) for i in range(12)], | ||
| 'target_names': ['weight', '_output_'], |
J-shang
reviewed
Apr 17, 2023
|
|
||
| if __name__ == "__main__": | ||
| fake_quantize() | ||
| evaluate() |
There was a problem hiding this comment.
remove if __name__ == "__main__":, this is a ipython style script.
ultmaster
approved these changes
Apr 17, 2023
J-shang
approved these changes
Apr 23, 2023
super-dainiu
pushed a commit
to super-dainiu/nni
that referenced
this pull request
May 27, 2023
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Test Options
Checklist
How to test