Joint Pruning, Quantization and Distillation for Transformers by vuiseng9 · Pull Request #1319 · openvinotoolkit/nncf

vuiseng9 · 2022-10-13T15:51:02Z

Changes

Add movement pruning feature, enabling end-to-end compression for transformer inference acceleration via OpenVINO

Reason for changes

Related tickets

Tests

Some basic tests are included, team will continue to refactor, add tests to this pull request.

ljaljushkin

Some initial comments

vuiseng9 · 2022-11-07T15:37:28Z

@yujiepan-work fyi #1354 and #1323 pertaining to the onnx export.

ljaljushkin

Overall, great job and test coverage! Only have some minor comments.

ljaljushkin · 2022-11-22T11:00:22Z

run pytorch pre-commit tests

* clean unnecessary codes * implement ada threshold with torch api rather than numpy api * pylint fix * simplify sparsity report * change loss calculation implementation * remove Enum.from_str * change logging level * add copyright * change scheduler param default value * rename * use single quote for strings * fix circulant import * add some docstrings * pylint fix

* documentation part1 * minor change

…sity

…ame example folder (#28)

* fix jit inplace * add softmax before checking onnx outputs * scheduler param dataclass * delete recipe "set" shortcut * rename * rename "linear_layer_sparsity"

…follow regularization warmup shape

* add test about "calc_init_threshold" called once * add tests about no weight override on model export * ensure compression loss is always torch.Tensor even if disabled * pylint fix * rename `sparse_layer` into `operand` * complete class and function docstrings * minor fix * add new lines to improve readability * use shared `nncf_caplog` instead of a self-implemented one * beautify `TestAutogradFunction` * use fixture to skip autograd tests when cuda is not available * delete markdown file dump for structured sparsity report * schema uses CONST_VARIABLE for default * change scheduler param order * refactor recipe (part1) * refactor recipe (part 2) * refactor recipe (part3) * refactor recipe (part4) * minor format changes * change training script path to a constant variable * rename "linear layer sparsity" in documentation * typo * change BuildingBlockType hash to hash(self.name) * add tests about compression state save & resume * fix wrong use of PTTensorListComparator in `test_can_load_state_dict` * fix bug that importance weights are still updated after warmup (due to adam characteristic) * bugfix in PTTensorListComparator in `test_can_load_state_dict` * rename `test_calc_init_threshold_called_once` * Split `TestModelSaving` to a standalone file * avoid triple ")" in codes for better readability * rename `structured_mask_context_list` to `structured_mask_contexts` * adjust argument name in `test_can_resume_training_from_compression_state`

vshampor · 2022-12-19T12:13:10Z

Missed a couple of my comments, but nothing critical at this stage. I have put in the approval.

* add missing docstring for `StructuredMaskContextGroup` and minor changes in other docstings * fix typo in documentation * add docstring for StructuredMaskContextStatistics * temporarily remove claim of fractional warmup start/end epoch

* pylint fix * documentation: require steps_per_epoch when warmup starts at "epoch in [0,1)" -> "at first epoch"

ljaljushkin · 2022-12-19T15:24:49Z

run tensorflow pre-commit tests

ljaljushkin

The latest minor changes in the documentation look good to me.
There was TF pre-commit failure, but seems like there's some sporadic issue. After re-run the validation is green.

…openvinotoolkit#1319)" This reverts commit 12351f8.

vuiseng9 requested a review from ljaljushkin October 13, 2022 15:51

github-actions Bot added NNCF Common Pull request that updates NNCF Common NNCF PT Pull requests that updates NNCF PyTorch labels Oct 13, 2022

vuiseng9 requested a review from AlexKoff88 October 13, 2022 15:52