Skip to content

Conversation

@QiliangCui
Copy link
Collaborator

@QiliangCui QiliangCui commented Aug 21, 2025

Description

Re-enable tests.
Add tpu_commons/tests/models/vllm back.

If the change fixes a bug or a Github issue, please include a link, e.g.,:
FIXES: b/438779127

Tests

Wait for CI Result.
https://buildkite.com/tpu-commons/tpu-commons-ci/builds/2100
https://buildkite.com/tpu-commons/tpu-commons-ci/builds/2116

@vllm-project vllm-project deleted a comment from github-actions bot Aug 21, 2025
@QiliangCui QiliangCui requested a review from hfan August 21, 2025 18:29
def generate_kv_caches(num_kv_heads, head_size, mesh, dtype):
cache_shape = get_kv_cache_shape_with_mesh(mesh, 1024, 16, num_kv_heads,
head_size, dtype)
head_size, t2j_dtype(dtype))
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hfan and @bythew3i , does this look ok?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the dtype change looks good to me

ref_output, _ = ref_ragged_paged_attention(
q,
jax.device_put(t2j(k), NamedSharding(mesh, P())),
jax.device_put(t2j(v), NamedSharding(mesh, P())),
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added this to avoid complaining "device mismatch". @bythew3i

Signed-off-by: Qiliang Cui <derrhein@gmail.com>
Copy link
Collaborator

@patemotter patemotter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, except I can't speak to the t2j_dtype(dtype)) question you asked.

@QiliangCui QiliangCui merged commit 30d7ce8 into main Aug 21, 2025
2 checks passed
Lumosis pushed a commit that referenced this pull request Aug 26, 2025
Signed-off-by: Qiliang Cui <derrhein@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants