deform_conv2d for mps

[DraBard]

🐛 Describe the bug

NotImplementedError: The operator torchvision::deform_conv2d is not currently implemented for the MPS device.

Versions

PyTorch version: 1.12.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.3 (x86_64)
GCC version: Could not collect
Clang version: 14.0.3 (clang-1403.0.22.14.1)
CMake version: Could not collect
Libc version: N/A

Python version: 3.10.9 (main, Mar 1 2023, 12:33:47) [Clang 14.0.6 ] (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M2

Versions of relevant libraries:
[pip3] flake8==6.0.0
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.5
[pip3] numpydoc==1.5.0
[pip3] torch==1.12.1
[conda] numpy 1.23.5 py310he50c29a_0
[conda] numpy-base 1.23.5 py310h992e150_0
[conda] numpydoc 1.5.0 py310hecd8cb5_0
[conda] pytorch 1.12.1 cpu_py310h64f2f56_1

[goldfishsound]

Yes, please add mps support for this ops.
Thank you :-)

[wtesler]

I also am interested in this

[goldfishsound]

I have been working on implementing MPS support for the deform_conv2d operator.
All implementation tests are passing, but critically, the assert_close in TestDeformConv::test_forward fails.
I have gone over my mps kernel and metal operator implementations countless times but just can't seem to nail the bug.
Also, I'm very new to open source and don't quite know how best to proceed.
Please let me know if anyone wants to get involved or just to point me in the right direction for next steps.
Thanks.

[wtesler]

@goldfishsound I'm excited to see this op working. How does the assertion fail? Is there more description on how the test fails? Can you compare the output? Are there any relevant links to the code or failed test that others can look at?

[wtesler]

I'm linking the op from the MPS tracker to this issue: PyTorch MPS Ops (view)

[goldfishsound]

@goldfishsound I'm excited to see this op working. How does the assertion fail? Is there more description on how the test fails? Can you compare the output? Are there any relevant links to the code or failed test that others can look at?

Thanks for your quick reply.

"How does the assertion fail?"
torch.testing.assert_close compares the result from the expected_fn function with the output of the operator and fails if the two results are too far appart, given a tolerance value for the parameters rtol and atol.

"Are there any relevant links to the code or failed test that others can look at?"
Yes. Can you advise me as how best to post the output of the tests? It's rather a lot and not sure if I just dump the raw output in a comment or via a document.
Do you recommend sharing a link to my working branch?

[bigcat88]

I have been working on implementing MPS support for the deform_conv2d operator. All implementation tests are passing, but critically, the assert_close in TestDeformConv::test_forward fails. I have gone over my mps kernel and metal operator implementations countless times but just can't seem to nail the bug. Also, I'm very new to open source and don't quite know how best to proceed. Please let me know if anyone wants to get involved or just to point me in the right direction for next steps. Thanks.

This is such exciting news, I hope you can do it, and thanks to your efforts we will finally be able to use Bria2.0 and BiRefNet models on macOS!

[mcioffi]

Interested as well, based on the release of briaai/RMBG-2.0

[goldfishsound]

Just dropped a PR:
Deform conv2d mps support #9026