[Misc] Add triton_kernels dependency

[varun-sundar-rabindranath]

Purpose

Add triton_kernels from https://github.com/triton-lang/triton/tree/main/python/triton_kernels as a dependency and pin it to tag v3.5.0.

Why v3.5.0:
triton_kernels is just a sub-directory in the Triton repo. vLLM supports Torch2.9 now and Torch2.7 ships with the Triton 3.5.0.

Why add triton_kernels as a dependency?
We use the matmul_ogs function from triton_kernels for mxfp4 fused_moe operations on Hopper. At the moment, this code-path is the fastest for running mxfp4 models on Hopper. At the moment, users have to install triton_kernels manually to access this code-path, with this, users can use it out-of-the-box.

Test Plan

Tried a fresh-build locally and executed,
TP : vllm serve openai/gpt-oss-120b --tensor-parallel-size 2 --no-enable-prefix-caching
DP : VLLM_ALL2ALL_BACKEND="deepep_high_throughput" vllm serve openai/gpt-oss-120b --data-parallel-size 2 --enable-expert-parallel --no-enable-prefix-caching

Test Result

On hopper, both commands defaults to using the Triton implementation for Mxfp4.
both commands produce reasonable gpt_oss eval metrics.

Solve issue #26582

[gemini-code-assist]

Code Review

This pull request adds triton_kernels as a dependency to support mxfp4 fused MoE operations. The change itself is correct in identifying the necessary package and version. However, there is a critical concern regarding the packaging for Docker. The new dependency is only added to requirements/cuda.txt, which may not be sufficient for it to be included in the final production Docker images. This could lead to runtime failures. I've added a comment with details on how to address this potential issue.

[varun-sundar-rabindranath]

cc @zyongye @mgoin @WoosukKwon @simon-mo

[varun-sundar-rabindranath]

BatchedOAITritonExperts was removed in PR #24588 and I missed removing it from the tests.
For context, eventhough matmul_ogs kernel from OpenAI Triton Kernels supports batched mode, it was removed as it is simply a dense gemm (does not mask invalid tokens) and not useful for the WideEP case.

[ywang96]

FYI - this is breaking the nightly installation

(new) coder@4xl40s devspaces (main) $ uv pip install vllm --extra-index-url https://wheels.vllm.ai/nightly
Using Python 3.12.11 environment at: new
  × Failed to resolve dependencies for `vllm` (v0.11.1rc4.dev17+g361a7463d.cu129)
  ╰─▶ Package `triton-kernels` was included as a URL dependency. URL dependencies must be expressed as direct requirements or constraints.
      Consider adding `triton-kernels @ git+https://github.com/triton-lang/triton.git@v3.5.0#subdirectory=python/triton_kernels` to your
      dependencies or constraints file.

[cjackal]

It seems this PR also blocks wheel install. (w/ the same uv pip install error message as above)

[mgoin]

@varun-sundar-rabindranath can we revert for now to unblock? We can make our own wheel or copy the kernels over

[varun-sundar-rabindranath]

Yes. I am reverting this now 👍

[varun-sundar-rabindranath]

PR to revert the requirement here #27659
cc @mgoin @cjackal @ywang96