not masking face with CUDA=12.8 and ONNXRuntime-GPU=1.22.0 using RTX 5090 32GB GPU

[m3zai6i]

Title: Face mask not applied correctly with CUDA 12.8 + ONNXRuntime-GPU 1.22.0 (RTX 5090)

Description:
Running the Deep Live Cam faceswap application with CUDA=12.8 and onnxruntime-gpu=1.22.0 on an RTX 5090 (32GB) results in incorrect masking. The face mask is not applied properly and causes constant flickering during inference.

Steps to Reproduce:

1. Run Deep Live Cam with ONNXRuntime-GPU.
2. Use RTX 5090 (32GB) and CUDA 12.8.
3. Observe that the mask flickers and does not correctly align with the face.

Expected Behavior:
The mask should be applied consistently without flickering.

Environment:

• GPU: RTX 5090 (32GB)
• CUDA: 12.8
• ONNXRuntime-GPU: 1.22.0
• Application: Deep Live Cam faceswap

[hacksider]

Do not use 1.22.0, just use the one in the instructions.

[m3zai6i]

In the requirements.txt, the dependency is listed as onnxruntime-gpu==1.22.0, while in the Installation Instructions (GPU Acceleration > CUDA Execution Provider), it suggests installing onnxruntime-gpu==1.21.0.

Neither of these versions work correctly on my setup with NVIDIA RTX 5090 (32GB VRAM) and Intel i9 14th Gen (96GB DDR5 RAM). The face mask flickers constantly and does not stay aligned on the face.

The only combination that works is:

• CUDA 11.8
• onnxruntime-gpu==1.16.0

But this results in only ~1.6 FPS, which is far below real-time performance, despite the high-end hardware.

Could you please provide guidance or an updated solution to ensure proper face masking and stable real-time performance on the latest hardware (RTX 5090 + CUDA 12.8 + onnxruntime-gpu)?

[Bionic711]

I am having a similar issue on an RTX 4090 24GB using onnxruntime-gpu==1.21.0 and tried 22 as well. Mask just constantly flickers. The bbox does a pretty good job of staying over the live face though in my case. CUDA12.8. Driver is 580.97. Also tried on driver 580.88.

[hacksider]

Do you have any recording of it?

[Bionic711]

@hacksider Let me know after you have reviewed it so I can remove the link to it.

[hacksider]

I just pinned the solution. Kindly check

[m3zai6i]

Do you have any recording of it?

Hi there, here is the link to the recording as requested of the application running on NVIDIA RTX 5090 (32GB) GPU, please check and provide an appropriate solution. So masking will be implemented in Real-Time.

Python = 3.10, 3.11
CUDA = 12.8
Onnxruntime-gpu = 1.21.0, 1.22.0

https://drive.google.com/file/d/1ZFQTvLbOmkwxe-2POqF0HJmTOrRg5sFJ/view?usp=drive_link

If you need more information, I'm happy to provide it.

[Bionic711]

Do you have any recording of it?

Hi there, here is the link to the recording as requested of the application running on NVIDIA RTX 5090 (32GB) GPU, please check and provide an appropriate solution. So masking will be implemented in Real-Time.

Python = 3.10, 3.11 CUDA = 12.8 Onnxruntime-gpu = 1.21.0, 1.22.0

https://drive.google.com/file/d/1ZFQTvLbOmkwxe-2POqF0HJmTOrRg5sFJ/view?usp=drive_link

If you need more information, I'm happy to provide it.

Same issue I had. I fixed it based on @hacksider recommendation. Here is the short of it.
Download this model.
https://github.com/facefusion/facefusion-assets/releases/download/models-3.0.0/inswapper_128_fp16.onnx
Replace the existing one in the models folder. Mine was about 1kb smaller than the one I downloaded. Fix it for me.

[m3zai6i]

Do you have any recording of it?

Hi there, here is the link to the recording as requested of the application running on NVIDIA RTX 5090 (32GB) GPU, please check and provide an appropriate solution. So masking will be implemented in Real-Time.
Python = 3.10, 3.11 CUDA = 12.8 Onnxruntime-gpu = 1.21.0, 1.22.0
https://drive.google.com/file/d/1ZFQTvLbOmkwxe-2POqF0HJmTOrRg5sFJ/view?usp=drive_link
If you need more information, I'm happy to provide it.

Same issue I had. I fixed it based on @hacksider recommendation. Here is the short of it. Download this model. https://github.com/facefusion/facefusion-assets/releases/download/models-3.0.0/inswapper_128_fp16.onnx Replace the existing one in the models folder. Mine was about 1kb smaller than the one I downloaded. Fix it for me.

@Bionic711 Hi, hope you are doing well.
Models I am using from the README of this repo.

GFPGANv1.4.pth (https://huggingface.co/hacksider/deep-live-cam/resolve/main/GFPGANv1.4.pth)
inswapper_128_fp16.onnx (https://huggingface.co/hacksider/deep-live-cam/resolve/main/inswapper_128_fp16.onnx)

Is the model (https://github.com/facefusion/facefusion-assets/releases/download/models-3.0.0/inswapper_128_fp16.onnx) you shared DIFFERENT?

[m3zai6i]

@Bionic711, @hacksider Hi, hope you are doing well.
Models I am using from the README of this repo.

GFPGANv1.4.pth (https://huggingface.co/hacksider/deep-live-cam/resolve/main/GFPGANv1.4.pth)
inswapper_128_fp16.onnx (https://huggingface.co/hacksider/deep-live-cam/resolve/main/inswapper_128_fp16.onnx)

Is the model (https://github.com/facefusion/facefusion-assets/releases/download/models-3.0.0/inswapper_128_fp16.onnx) you shared DIFFERENT?

[Bionic711]

@m3zai6i It is the "same" model, but as I stated, the one I downloaded was slightly larger than the one that was previously downloaded. Once I replaced it, my issue, which looked like yours, was resolved. I would try it anyway, what do you have to lose? My current one on disk (the new downloaded one from the link), shows in Windows as 271,173KB. The first one I had was 270k KB.

[m3zai6i]

@Bionic711, thanks for your response. I updated the model using your link. Now, the flickering issue is resolved, but getting only 15FPS on the RTX 5090 GPU (32GB VRAM). How many FPS are you getting?

[Bionic711]

On my 4090, I am showing between 14 and 16 fps.

[m3zai6i]

@Bionic711 @hacksider
Currently, I'm getting 18.9 FPS on RTX 5090
Python 3.11
Cuda = 12.8

How to get real-time performance like 25 to 30+ FPS? Any solutions or suggestions?