remove attn output view kernel

[BoyuanFeng]

Before this PR, attention output is allocated and initialized with 0 (due to torch.zeros), and the view into a shape, before the output tensor is used by any other ops. This becomes a triton kernel of ~1 us latency, which is on-par with a rope/layer norm (~1.6us) latency.

This PR changes to allocate with torch.empty which only allocates the tensor and does not initialize it. This allocation will be removed by cudagraph so it is free.

As a result, this PR removes the attn_out_view kernel at the end of this qwen3-0.6b trace.

See #26682 (comment) for perf win.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @BoyuanFeng.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[gemini-code-assist]

Code Review

This pull request optimizes attention output tensor allocation by using torch.empty instead of torch.zeros, avoiding an unnecessary kernel launch and improving performance. A new configuration init_attn_out is introduced to maintain backward compatibility for attention backends that require zero-initialized output tensors. The changes are logical and well-motivated. I have one suggestion to improve code clarity and maintainability in vllm/attention/layer.py by refactoring the output tensor shape calculation to avoid variable reuse, which can be error-prone.

[ProExpertProg]

cc @WoosukKwon I think you added the zeroinit attention?

[WoosukKwon]

@BoyuanFeng @ProExpertProg

I think it was my mistake that we zero-initialize the buffer for every forward pass. My intent was to do it only when the whole attention operation is skipped, like profiling run. I think we can move output.zero_() to here?

vllm/vllm/v1/attention/backends/flash_attn.py

Lines 488 to 490 in 314285d

	if attn_metadata is None:
	# Profiling run.
	return output

I think it's still important to keep the buffers away from NaN, because some kernels could potentially err with it. Previously, I met this issue with a custom MoE kernel whose top-k and routing kernel doesn't handle NaNs well.

[BoyuanFeng]

@WoosukKwon thanks for the info! I added output.fill_(0) for profiling runs and use torch.empty for other fwd. I checked that this also removes the extra triton kernel.

[ProExpertProg]

Can you just check the attention fusion pass? I think it might rely on the fact that we use zeros for attention output