fix block-size description

[chenqianfzh]

Per vllm official doc in https://docs.vllm.ai/en/stable/models/engine_args.html, the parameter of --block-size can take the value of {8,16,32,64,128}. However, I hit error "RuntimeError: Unsupported block size: 128" when I tried out "--block-size 128"

I hunted the code and saw the following logic in csrc/attention/paged_attention_v2.cu:

#define CALL_V2_LAUNCHER_BLOCK_SIZE(T, CACHE_T, KV_DTYPE)         \
  switch (block_size) {                                           \
    case 8:                                                       \
      CALL_V2_LAUNCHER_SPARSITY(T, CACHE_T, 8, KV_DTYPE);         \
      break;                                                      \
    case 16:                                                      \
      CALL_V2_LAUNCHER_SPARSITY(T, CACHE_T, 16, KV_DTYPE);        \
      break;                                                      \
    case 32:                                                      \
      CALL_V2_LAUNCHER_SPARSITY(T, CACHE_T, 32, KV_DTYPE);        \
      break;                                                      \
    default:                                                      \
      TORCH_CHECK(false, "Unsupported block size: ", block_size); \
      break;                                                      \
  }

Similar logic is also seen in csrc/attention/paged_attention_v1.cu.

Updated the parameter allowed value per the real logic.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[kzawora-intel]

Hey, while GPU kernels might not support block sizes greater than 32, other accelerators do. On HPU, going below block size 128 is very detrimental to performance. I don't think the option to use greater block sizes should be removed, rather a proper assertion and log should be added to verify_args. I've created PR #11259 reverting this one.

[chenqianfzh]

Hey, while GPU kernels might not support block sizes greater than 32, other accelerators do. On HPU, going below block size 128 is very detrimental to performance. I don't think the option to use greater block sizes should be removed, rather a proper assertion and log should be added to verify_args. I've created PR #11259 reverting this one.

Thanks!