Skip to content

[CK] Fix divide-by-zero crash for grouped conv kernels - #6132

Merged
bartekxk merged 3 commits into
developfrom
crash_pytorch_test_due_to_CK
Apr 23, 2026
Merged

[CK] Fix divide-by-zero crash for grouped conv kernels#6132
bartekxk merged 3 commits into
developfrom
crash_pytorch_test_due_to_CK

Conversation

@k-artem

@k-artem k-artem commented Apr 2, 2026

Copy link
Copy Markdown
Contributor

Motivation

During run pytorch unit tests for conv3d: test_dtypes_nn_functional_conv3d_cuda, test_fake_crossref_backward_amp_nn_functional_conv3d_cuda_float32
found divide-by-zero crash during CK kernel selection.

Refs ROCM-20764

Technical Details

Add assert for K0PerBlock equal 0, also covered other potential places related with k_batch calculation.

Test Plan

Run miopen command extracted from mentioned test:
MIOpenDriver convfp16 --spatial_dim 3 -I NCDHW -O NCDHW -f NCDHW -n 1 -c 1 -k 1 -g 1 --in_d 4 -H 4 -W 4 --fil_d 4 -y 4 -x 4 --pad_d 0 -p 0 -q 0 --conv_stride_d 2 -u 2 -v 2 --dilation_d 1 -l 1 -j 1 -m conv -F 4 -t 1

Test Result

Passed

Submission Checklist

k-artem added 3 commits April 2, 2026 20:45
Details:
Detected in ROCM-20764 during run pytorch unit tests
for conv3d with 0 padding.

Signed-off-by: Artem Kuzmitckii <artem.kuzmitckii@amd.com>
@k-artem

k-artem commented Apr 17, 2026

Copy link
Copy Markdown
Contributor Author

@ROCm/ck-reviewers please have a look

@bartekxk
bartekxk merged commit e97065d into develop Apr 23, 2026
32 checks passed
@bartekxk
bartekxk deleted the crash_pytorch_test_due_to_CK branch April 23, 2026 20:10
assistant-librarian Bot pushed a commit to ROCm/composable_kernel that referenced this pull request Apr 23, 2026
[CK] Fix divide-by-zero crash for grouped conv kernels
 (#6132)

## Motivation

During run pytorch unit tests for conv3d:
`test_dtypes_nn_functional_conv3d_cuda`,
`test_fake_crossref_backward_amp_nn_functional_conv3d_cuda_float32`
found divide-by-zero crash during CK kernel selection.

Refs ROCM-20764

## Technical Details

Add assert for K0PerBlock equal 0, also covered other potential places
related with k_batch calculation.

## Test Plan
Run miopen command extracted from mentioned test:
`MIOpenDriver convfp16 --spatial_dim 3 -I NCDHW -O NCDHW -f NCDHW -n 1
-c 1 -k 1 -g 1 --in_d 4 -H 4 -W 4 --fil_d 4 -y 4 -x 4 --pad_d 0 -p 0 -q
0 --conv_stride_d 2 -u 2 -v 2 --dilation_d 1 -l 1 -j 1 -m conv -F 4 -t
1`
## Test Result
Passed

## Submission Checklist

- [X] Look over the contributing guidelines at
https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

Signed-off-by: Artem Kuzmitckii <artem.kuzmitckii@amd.com>
aledudek pushed a commit that referenced this pull request May 20, 2026
## Motivation

During run pytorch unit tests for conv3d:
`test_dtypes_nn_functional_conv3d_cuda`,
`test_fake_crossref_backward_amp_nn_functional_conv3d_cuda_float32`
found divide-by-zero crash during CK kernel selection.

Refs ROCM-20764

## Technical Details

Add assert for K0PerBlock equal 0, also covered other potential places
related with k_batch calculation.

## Test Plan
Run miopen command extracted from mentioned test:
`MIOpenDriver convfp16 --spatial_dim 3 -I NCDHW -O NCDHW -f NCDHW -n 1
-c 1 -k 1 -g 1 --in_d 4 -H 4 -W 4 --fil_d 4 -y 4 -x 4 --pad_d 0 -p 0 -q
0 --conv_stride_d 2 -u 2 -v 2 --dilation_d 1 -l 1 -j 1 -m conv -F 4 -t
1`
## Test Result
Passed

## Submission Checklist

- [X] Look over the contributing guidelines at
https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

Signed-off-by: Artem Kuzmitckii <artem.kuzmitckii@amd.com>
shumway pushed a commit to ROCm/composable_kernel that referenced this pull request May 27, 2026
[CK] Fix divide-by-zero crash for grouped conv kernels (#6132)

## Motivation

During run pytorch unit tests for conv3d:
`test_dtypes_nn_functional_conv3d_cuda`,
`test_fake_crossref_backward_amp_nn_functional_conv3d_cuda_float32`
found divide-by-zero crash during CK kernel selection.

Refs ROCM-20764

## Technical Details

Add assert for K0PerBlock equal 0, also covered other potential places
related with k_batch calculation.

## Test Plan
Run miopen command extracted from mentioned test:
`MIOpenDriver convfp16 --spatial_dim 3 -I NCDHW -O NCDHW -f NCDHW -n 1
-c 1 -k 1 -g 1 --in_d 4 -H 4 -W 4 --fil_d 4 -y 4 -x 4 --pad_d 0 -p 0 -q
0 --conv_stride_d 2 -u 2 -v 2 --dilation_d 1 -l 1 -j 1 -m conv -F 4 -t
1`
## Test Result
Passed

## Submission Checklist

- [X] Look over the contributing guidelines at
https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

Signed-off-by: Artem Kuzmitckii <artem.kuzmitckii@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants