Dygraph post trainging quantization #33445
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
XGZhang11
approved these changes
Jun 21, 2021
XieYunshen
approved these changes
Jun 21, 2021
|
|
||
| class ImperativePTQ(object): | ||
| """ | ||
| Applying static post_training quantization to the dgraph model. |
There was a problem hiding this comment.
有没有更专业的词来表示dgraph model？
Comment on lines
+42
to
+44
| algo(str): The algorithm in post_training quantizaion to be used. | ||
| activation_bits(int): quantization bit number for activations. | ||
| weight_bits(int): quantization bit number for weights. |
There was a problem hiding this comment.
这些参数并没有出现在__init__参数列表中。
另外，需要详细说明下默认情况下input和weight各用的是什么quantizer。
| Add hook to the leaf layer to calculate the threshold of inputs and outputs. | ||
| Args: | ||
| model(paddle.nn.Layer): The model to be quantized. |
| Args: | ||
| model(paddle.nn.Layer): The model to be quantized. | ||
| Returns: | ||
| None |
There was a problem hiding this comment.
如果inplace == False的话，是不是return一个model比较好？
我看最后一行有return model，这里需要更新下。
| assert isinstance(model, paddle.nn.Layer), \ | ||
| "The model must be the instance of paddle.nn.Layer." | ||
|
|
||
| if not inplace: |
There was a problem hiding this comment.
什么情况下需要inplace==False呢
There was a problem hiding this comment.
如果需要保留原始输入FP32模型，用于和量化模型进行对比精度，可以直接设置inplace=False
|
|
||
| hook = ptq_hooks.quant_forward_post_hook | ||
| hook_handle = layer.register_forward_post_hook(hook) | ||
| quant_config.hook_handle = hook_handle |
There was a problem hiding this comment.
后续会不会添加更多的hook？所以hook_handle这个名字是不是可以叫『quant_forward_post_hook』？
| hook = ptq_hooks.quant_forward_post_hook | ||
| hook_handle = layer.register_forward_post_hook(hook) | ||
| quant_config.hook_handle = hook_handle | ||
| layer._forward_post_hooks.move_to_end( |
There was a problem hiding this comment.
在调用统计激活的hook之前，有没有对weight做量化操作？
| The PTQ config shows how to quantize the inputs and outputs. | ||
| """ | ||
|
|
||
| def __init__(self, activation_quantizer, weight_quantizer): |
| return expanded_quantized_bins | ||
|
|
||
|
|
||
| def safe_entropy(reference_distr_P, P_sum, candidate_distr_Q, Q_sum): |
There was a problem hiding this comment.
这些方法在静态图PTQ中没被用到么？不能直接复用？
| @@ -0,0 +1,288 @@ | |||
| # copyright (c) 2018 paddlepaddle authors. all rights reserved. | |||
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
New features
PR changes
Others
Describe
Dygraph post trainging quantization