feat: introduce USE_NPU_TORCH flag for debugging and enhance NPU support for Qwen3-Dense[4/N]. #590

yingxudeng · 2025-12-23T16:06:02Z

No description provided.

…alLMImplBase.

yingxudeng · 2025-12-23T16:09:21Z

This PR cannot be merged/tested until #572 #563 and #583 is landed.

Please focus your review only on the commit shown in the image above. All other commits can be ignored for now. Once the dependent PRs are merged, I will rebase the branch to clean up the history.

xllm/core/layers/common/tests/tests_utils.h

xllm/core/layers/common/attention_metadata.h

xllm/core/layers/common/qwen2_5_vision_layer.h

xllm/core/layers/common/indexer.h

xllm/core/layers/common/qwen2_decoder_layer.h

RobbieLeung · 2025-12-24T04:35:01Z

xllm/core/layers/common/rotary_embedding.cpp

-  cos_ = cos_sin_vec[0].view({-1, rotary_dim});
-  sin_ = cos_sin_vec[1].view({-1, rotary_dim});
+  cos_ = cos_sin_vec[0].reshape({-1, rotary_dim});
+  sin_ = cos_sin_vec[1].reshape({-1, rotary_dim});


Some operation will change origin tensor?

xllm/core/layers/npu_torch/attention.cpp

xllm/core/framework/model/causal_vlm.h

…ort for Qwen3-Dense.

yingxudeng added 5 commits December 18, 2025 16:33

refactor: rename layer and model interfaces for torch_npu preparation.

8970079

refactor: update destructors to use default implementation.

bb2d192

feat: add wrapper for torch_npu layers.

22bbe37

refactor: clean up conditional compilation directives in model headers.

0e40473

refactor: update causal LM implementations to inherit from LlmForCaus…

4ba5d78

…alLMImplBase.

yingxudeng requested review from liutongxuan and yq33victor December 23, 2025 16:06

yingxudeng changed the title ~~feat: introduce USE_NPU_TORCH flag for debugging and enhance NPU support for Qwen3-Dense.~~ feat: introduce USE_NPU_TORCH flag for debugging and enhance NPU support for Qwen3-Dense[4/N]. Dec 23, 2025

yingxudeng force-pushed the feat/npu_backend_torch_3_4_20251223 branch from 43a1e2d to 3d505b6 Compare December 24, 2025 03:06