- 
                Notifications
    
You must be signed in to change notification settings  - Fork 21
 
[RpaV3] Renable Tests Part 1 #534
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
51e50a9    to
    abf834f      
    Compare
  
    | def generate_kv_caches(num_kv_heads, head_size, mesh, dtype): | ||
| cache_shape = get_kv_cache_shape_with_mesh(mesh, 1024, 16, num_kv_heads, | ||
| head_size, dtype) | ||
| head_size, t2j_dtype(dtype)) | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the dtype change looks good to me
| ref_output, _ = ref_ragged_paged_attention( | ||
| q, | ||
| jax.device_put(t2j(k), NamedSharding(mesh, P())), | ||
| jax.device_put(t2j(v), NamedSharding(mesh, P())), | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added this to avoid complaining "device mismatch". @bythew3i
Signed-off-by: Qiliang Cui <derrhein@gmail.com>
abf834f    to
    b288d0e      
    Compare
  
    There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, except I can't speak to the t2j_dtype(dtype)) question you asked.
Signed-off-by: Qiliang Cui <derrhein@gmail.com>
Description
Re-enable tests.
Add
tpu_commons/tests/models/vllmback.If the change fixes a bug or a Github issue, please include a link, e.g.,:
FIXES: b/438779127
Tests
Wait for CI Result.
https://buildkite.com/tpu-commons/tpu-commons-ci/builds/2100
https://buildkite.com/tpu-commons/tpu-commons-ci/builds/2116