Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: bug where some specific shapes make inference on cuda graphs crashing #177

Merged
merged 7 commits into from
Nov 15, 2022

Conversation

pommedeterresautee
Copy link
Member

@pommedeterresautee pommedeterresautee commented Nov 13, 2022

fix #176
Also remove the need for a workaround regarding 32x32 issue, etc. -> new shapes have been added in the test suite.

@pommedeterresautee pommedeterresautee self-assigned this Nov 13, 2022
@pommedeterresautee pommedeterresautee added bug Something isn't working benchmark Measure, measure, measure labels Nov 13, 2022
@github-actions github-actions bot added fix hurrah, bug fixed! and removed fix hurrah, bug fixed! labels Nov 13, 2022
@pommedeterresautee pommedeterresautee added triton triton kernel / implementation and removed bug Something isn't working benchmark Measure, measure, measure labels Nov 13, 2022
@pommedeterresautee
Copy link
Member Author

test pass


=========================================================================================================== warnings summary ===========================================================================================================
test/test_torchdynamo.py: 30 warnings
  /home/geantvert/.local/share/virtualenvs/kernl/lib/python3.9/site-packages/torch/jit/_check.py:181: UserWarning: The TorchScript type system doesn't support instance-level annotations on empty non-base types in `__init__`. Instead, either 1) use a type annotation in the class body, or 2) wrap the type in `torch.jit.Attribute`.
    warnings.warn("The TorchScript type system doesn't support "

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
====================================================================================== 2684 passed, 26 skipped, 30 warnings in 9000.74s (2:30:00) ======================================================================================

@github-actions github-actions bot added fix hurrah, bug fixed! and removed fix hurrah, bug fixed! labels Nov 14, 2022
@pommedeterresautee
Copy link
Member Author

following new changes:

pytest test/test_torchdynamo.py -k "dynamo_optimized and cuda and benchmark"
...
======================================================================================= 52 passed, 214 deselected in 1013.59s (0:16:53) =======================================================================================

also tested and pass:

pytest test/test_attention.py -k "triton and fp16 and non-causal and benchmark and not no_cuda_graphs"   

@pommedeterresautee pommedeterresautee merged commit 9c3ec9b into main Nov 15, 2022
@pommedeterresautee pommedeterresautee deleted the fix/shape_crash_cuda_graphs branch November 15, 2022 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fix hurrah, bug fixed! triton triton kernel / implementation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix crash on T5 inference for some specific shapes
2 participants