[Kernel] Faster pre-processing time for W4A8

[czhu-cohere]

Purpose

The W4A8 kernels require a special encoding/pre-processing step (one for weights, one for scales). The current CUTLASS implementation walks the values 1-by-1 on CPU which is slow, causing startup to take ~10 minutes for a 100B model.

The simple CUDA implementation is much faster for this specific op and reduce total model load time from ~10min to <1min. The main logic is to build a lookup table to map 8-bit nibbles (2 4-bit values) at a time, where 1..7 -> (8 - v) and 0, 8..15 are untouched. (A similar thing could be implemented for the scales pre-processing but that is less of a bottleneck than weights, and waiting <1min on startup seems reasonable)

Test Plan

Correctness - the existing test here pytest tests/kernels/quantization/test_cutlass_w4a8.py should cover it (if the encoding step is wrong the result will be wrong)

Speedup - look at logs for vllm serve and compare the startup time for Command A (111B params)

E2E - compare gsm8k lm-eval before and after

Test Result

pytest tests/kernels/quantization/test_cutlass_w4a8.py 
======================================================================== test session starts =========================================================================
platform linux -- Python 3.12.11, pytest-8.4.1, pluggy-1.6.0
rootdir: /root/vllm
configfile: pyproject.toml
plugins: anyio-4.10.0
collected 121 items                                                                                                                                                  

tests/kernels/quantization/test_cutlass_w4a8.py .............................................................................................................. [ 90%]
...........                                                                                                                                                    [100%]

======================================================================== 121 passed in 52.64s ========================================================================

startup time

before
[gpu_model_runner.py:2018] Model loading took 62.5527 GiB and 558.104425 seconds

after
[gpu_model_runner.py:1980] Model loading took 62.5522 GiB and 44.892353 seconds

lm-eval gsm8k

before: 0.8627748294
after: 0.862774829

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[mgoin]

Nice and clean, thanks for the work!

[czhu-cohere]

@mgoin Thanks for the review, I think the failure is unrelated, OK to merge?