cherry pick Habana software v1.23.0

[xin3he]

PR Type

Enhancement, Bug fix

---

Description

• Added QuantizedHpuBlockSoftmaxConstMax class for handling block softmax operations

• Updated scale calculation for FSDPA to prevent division by zero

• Changed dynamic quant check to use op string instead of type

• Fixed scale calculation flow for CGUID in weight scaling

---

Diagram Walkthrough

flowchart LR
  A["Add QuantizedHpuBlockSoftmaxConstMax"] -- "Handle block softmax" --> B["Update FSDPA scale calculation"]
  B -- "Prevent division by zero" --> C["Change dynamic quant check"]
  C -- "Use op string" --> D["Fix CGUID scale calculation"]

Loading

File Walkthrough

	Relevant files
Enhancement	1 files hpu_quantized_func_wrapper.py Add Block Softmax and Update Scale Calculation                      +23/-11
Bug fix	1 files quantize.py Update Dynamic Quant Check and Op String Usage                      +2/-2
Additional files	16 files common.py +5/-0      external_func_impl.py +40/-0    fp_utils.py +3/-3      patching_common.py +7/-1      quantized_func_wrapper.py +1/-0      xpu_quantized_func_wrapper.py +4/-4      scale.py +1/-1      scale_handler.py +4/-0      ops_quantizer.py +27/-21  round_scales_function.py +2/-2      scales_method.py +52/-45  utils.py +1/-1      vllm_functions.py +0/-32    helper_modules.py +197/-50 quant_config.py +4/-4      test_xpu_basic.py +79/-8

---

[PRAgent4INC]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Naming Consistency The new functions calculate_scale_maxabs_with_cguid and calculate_scale_rounding_with_cguid have names that include cguid, but the old functions calculate_scale_maxabs and calculate_scale_rounding do not. Ensure that the naming is consistent or that the addition of cguid is justified and documented. def calculate_scale_maxabs_with_cguid(x, maxMode, **kwargs): return torch.ops.hpu.calculate_scale_for_cast( x, maxMode.value, ScaleCalculationRoundingMode.NO_SCALE_ROUNDING.value, **kwargs ) def calculate_scale_rounding_with_cguid(x, scaleMode, **kwargs): return torch.ops.hpu.calculate_scale_for_cast( x, ScaleCalculationMaxMode.NO_MAX_CALCULATION.value, scaleMode.value, **kwargs ) Function Renaming Impact The renaming of calc_maxabs_scale to calc_scale_from_maxabs could affect other parts of the codebase that rely on the original function name. Verify that all references to the old function name have been updated accordingly. def calc_scale_from_maxabs(xmaxabs, fullscale, backoff=1): scale = xmaxabs / (fullscale * backoff) return scale

[PRAgent4INC]

Failed to generate code suggestions for PR

[xin3he]

Hi @linoybu Feel free to drop any review comments! 😀

[yiliu30]

LGTM

[thuang6]

why this "[SW-239679] temporary fix for static quant test (#298)" commit has no file change?

[xin3he]

why this "[SW-239679] temporary fix for static quant test (#298)" commit has no file change?

https://github.com/habana-internal/neural-compressor-fork/pull/298
Thanks for raising that. the cherry-pick is empty because that one fix is already in INC, and another file: test_autoround.py was moved to another place now.
The cherry-pick is not finished yet. Please expect more changes happen and hopefully we can enable the test_autoround.py back.