[Bug]: Bug in quantization/awq /gemm_kernels.cu gemm_forward_4bit_cuda_m16nXk32 More result have been write

[mengsoso]

Your current environment

The output of `python collect_env.py`

Your output of `python collect_env.py` here

🐛 Describe the bug

When N=64, we don't have 4*8=32 c_warp result; In this case, we only have 2(N/32) * 8=16 c_warp results.

[youkaichao]

cc @mgoin

[mgoin]

It looks like this was introduced during the refactor here #2723

Essentially no changes to the kernel have been made since, only formatting or removing warnings. @mengsoso do you have a model/scenario where this bug occurs?

[mengsoso]

@mgoin thank you for response
When we evaluate the result only for the gemm_forward_4bit_cuda_m16nXk32 kernel, the result is wrong when N=64.

reference #2723
When N=64 / 128：

① Parameter assignment：

② calculate：

③ Result write back：

When N=64, you can always only operate 2 (N/32) *8 c_warp.

[casper-hansen]

Good catch. Here is a model with group size 64, for reference.

https://huggingface.co/TechxGenus/DeepSeek-Coder-V2-Lite-Instruct-AWQ