V1 for fp4

[maleksan85]

commands:

HIP_VISIBLE_DEVICES=3 VLLM_USE_V1=1 VLLM_V1_USE_PREFILL_DECODE_ATTENTION=1 \
VLLM_TRITON_FP4_GEMM_USE_ASM=1 \
AMDGCN_USE_BUFFER_OPS=1 \
VLLM_USE_AITER_TRITON_ROPE=1 \
\
VLLM_USE_AITER_TRITON_SILU_MUL=0 \
\
TRITON_HIP_ASYNC_COPY_BYPASS_PERMUTE=1 \
TRITON_HIP_USE_ASYNC_COPY=1 \
TRITON_HIP_USE_BLOCK_PINGPONG=1 \
TRITON_HIP_ASYNC_FAST_SWIZZLE=1 \
TRITON_HIP_PRESHUFFLE_SCALES=1 \
VLLM_ROCM_USE_AITER=1 \
VLLM_ROCM_USE_AITER_MHA=0 \
VLLM_ROCM_USE_AITER_PAGED_ATTN=1 \
VLLM_ROCM_USE_AITER_RMSNORM=1 \
python /data/scripts/llm_test.py --model /data/Llama-3.1-405B-Instruct-wmxfp4-amxfp4-kvfp8-scale-uint8 --prompt "who am I?" \
    --tensor-parallel-size 1 \
    --max-model-len 4112 \
    --dtype auto \
    --max-num-batched-tokens 4112 \
    --gpu-memory-util 0.99 \
    --no-enable-prefix-caching \
    --max-num-seqs 128 \
    --max-seq-len-to-capture 4112 \
    --kv-cache-dtype fp8 \
    --compilation-config '{"full_cuda_graph":true, "cudagraph_capture_sizes":[1]}'

Generated: I am a 5 letter word. If you take away my first letter, I still sound the same. If you take away my last letter, I still sound the same. But remove my middle letter and I do not sound the same. What am I? I am "Swims". When you remove my first letter "S", I am still pronounced the same way, as "wims". When you remove my last letter "S", I am still pronounced the same way, as "swim". But when you remove my middle letter "M", I am pronounced differently, as "swis". Am I correct?