Support for NVIDIA Blackwell Architecture (RTX 50-series, sm_120) - Capability (12, 0) too new

[Pr3zLy]

I am trying to run inference using diffusers with enable_xformers_memory_efficient_attention() on a Windows machine equipped with the new NVIDIA RTX 5070 Ti (Blackwell architecture).

The execution crashes with a NotImplementedError because xformers does not yet recognize or support Compute Capability 12.0 (sm_120).

Environment
OS: Windows 10/11

GPU: NVIDIA GeForce RTX 5070 Ti

Python: 3.10

Error Log
Plaintext

NotImplementedError: No operator found for memory_efficient_attention_forward with inputs:
query : shape=(1, 2, 1, 40) (torch.float32)
key : shape=(1, 2, 1, 40) (torch.float32)
value : shape=(1, 2, 1, 40) (torch.float32)
attn_bias : <class 'NoneType'>
p : 0.0
fa3F@2.8.3-133-gde1584b is not supported because:
requires device with capability <= (9, 0) but your GPU has capability (12, 0) (too new)
dtype=torch.float32 (supported: {torch.bfloat16, torch.float16})
requires device with capability == (8, 0) but your GPU has capability (12, 0) (too new)
cutlassF-pt is not supported because:
requires device with capability <= (9, 0) but your GPU has capability (12, 0) (too new)
Additional Context
Since PyTorch Stable (2.1.x - 2.5.x) does not support the RTX 50-series yet, users with this hardware must use PyTorch Nightly. It seems xformers kernels need to be updated to allow execution on sm_120 devices, or at least fallback gracefully.

[huai-chang]

Thanks for the detailed report and the analysis regarding the RTX 50-series! This is very helpful context.

[huai-chang]

Since the RTX 50 series is very new, xformers hasn't been updated to support sm_120 yet.

As a temporary workaround, could you try disabling xformers? You can simply comment out these lines in the code:

• online inference
  

  PersonaLive/src/wrapper.py

  Line 148 in 57b927f

  self.enable_xformers_memory_efficient_attention()

  
  

  PersonaLive/src/wrapper_trt.py

  Line 129 in 57b927f

  self.enable_xformers_memory_efficient_attention()

• offline inference
  

  PersonaLive/inference_offline.py

  Lines 124 to 130 in 57b927f

  	if is_xformers_available():
  	reference_unet.enable_xformers_memory_efficient_attention()
  	denoising_unet.enable_xformers_memory_efficient_attention()
  	else:
  	raise ValueError(
  	"xformers is not available. Make sure it is installed correctly"
  	)

This should bypass the error and let the inference run without issues (albeit slightly slower).

---

• [2025.12.17] 🛠️ Update: Automatically skip xformers when unavailable to prevent crashes.

[huai-chang]

I get this error after that:

(personalive) C:\AI\PersonaLive>python inference_offline.py C:\Users\Jasper\anaconda3\envs\personalive\lib\site-packages\transformers\utils\generic.py:441: FutureWarning: torch.utils._pytree._register_pytree_nodeis deprecated. Please usetorch.utils._pytree.register_pytree_nodeinstead. _torch_pytree._register_pytree_node( C:\Users\Jasper\anaconda3\envs\personalive\lib\site-packages\transformers\utils\generic.py:309: FutureWarning:torch.utils._pytree._register_pytree_nodeis deprecated. Please usetorch.utils._pytree.register_pytree_nodeinstead. _torch_pytree._register_pytree_node( C:\Users\Jasper\anaconda3\envs\personalive\lib\site-packages\diffusers\models\dual_transformer_2d.py:20: FutureWarning:DualTransformer2DModelis deprecated and will be removed in version 0.29. ImportingDualTransformer2DModelfromdiffusers.models.dual_transformer_2dis deprecated and this will be removed in a future version. Please usefrom diffusers.models.transformers.dual_transformer_2d import DualTransformer2DModel, instead. deprecate("DualTransformer2DModel", "0.29", deprecation_message) device cuda Some weights of the model checkpoint were not used when initializing UNet2DConditionModel: ['conv_norm_out.weight, conv_norm_out.bias, conv_out.weight, conv_out.bias'] Traceback (most recent call last): File "C:\AI\PersonaLive\inference_offline.py", line 254, in <module> main(args) File "C:\AI\PersonaLive\inference_offline.py", line 128, in main raise ValueError( ValueError: xformers is not available. Make sure it is installed correctly

if you run the inference_offline.py, you can comment out these lines:

PersonaLive/inference_offline.py

Lines 124 to 130 in 57b927f

	if is_xformers_available():
	reference_unet.enable_xformers_memory_efficient_attention()
	denoising_unet.enable_xformers_memory_efficient_attention()
	else:
	raise ValueError(
	"xformers is not available. Make sure it is installed correctly"
	)

[huai-chang]

Great, offline works. Online still has some issue's, might be that i wrongly installed something but let me know:

`(personalive) C:\AI\PersonaLive>python inference_online.py
...

It looks like a TensorRT issue.

You can try disabling TensorRT by changing use_trt to False in

PersonaLive/inference_online.py

Line 27 in 57b927f

use_trt = True

If you do want to use TensorRT for acceleration, you would need to run torch2trt.py first to convert the models. However, please note that the conversion script seems to have some issues on Windows. #8

[merecesarchviz]

Great, offline works. Online still has some issue's, might be that i wrongly installed something but let me know:
`(personalive) C:\AI\PersonaLive>python inference_online.py
...

It looks like a TensorRT issue.

You can try disabling TensorRT by changing use_trt to False in

PersonaLive/inference_online.py

Line 27 in 57b927f

use_trt = True
If you do want to use TensorRT for acceleration, you would need to run torch2trt.py first to convert the models. However, please note that the conversion script seems to have some issues on Windows. #8

in my case it resolve that error but now in end when it give us the server link to open!! it dont open in browser!

[huai-chang]

Maybe you can try http://localhost:7860. 🤔

[merecesarchviz]

@huai-chang using localhost:7860 it open´s now the server page!

but now another issue appear no camera connection! how to connect the camera in the ui?

[huai-chang]

@huai-chang using localhost:7860 it open´s now the server page!

but now another issue appear no camera connection! how to connect the camera in the ui?

To start the camera feed, please follow these steps:

1. Click Fuse Reference.

2. Click Start Animation.

The webpage should then start acquiring the camera data.

[merecesarchviz]

@huai-chang using localhost:7860 it open´s now the server page!
but now another issue appear no camera connection! how to connect the camera in the ui?

To start the camera feed, please follow these steps:

1. Click Fuse Reference.
2. Click Start Animation.

The webpage should then start acquiring the camera data.

it open´s the camera, but no animation show in the right side! some error popup

[merecesarchviz]

@huai-chang I did pip install websockets and run again now all works perfect! the only issue i see is the realtime is very slow responsure, that happens why?

[dknos]

⚡ Guide: Running on Windows with RTX 50-Series (Blackwell)

I got it working after 2 days of debugging. Very slow without TensorRT but will try installing that next. Here is an AI summary of my conversations with LLM's.

If you are using an RTX 5080/5090 on Windows, you will likely encounter CUDA error: no kernel image or avformat.lib crashes because the current requirements are built for Linux/Older GPUs.

Here is the "Golden Path" to get this running in ~5 minutes.

1. Create a clean environment

Do not install the default requirements.

conda create -n personalive python=3.10 -y
conda activate personalive

2. Install PyTorch Nightly (Required for Blackwell)

Standard PyTorch does not support sm_120 yet. You must use the nightly build.

pip install --pre torch torchvision torchaudio --index-url [https://download.pytorch.org/whl/nightly/cu128](https://download.pytorch.org/whl/nightly/cu128)

3. Install Windows-Safe Dependencies

This installs necessary libs but skips the Linux-only packages (pycuda, tensorrt) and installs a pre-built binary for av.

pip install accelerate decord diffusers==0.27.0 einops fastapi huggingface-hub markdown2 mediapipe "numpy<2" omegaconf onnx onnxruntime opencv-python-headless pillow scikit-image starlette tqdm transformers uvicorn peft python-multipart colored websockets av

4. Auto-Patch: Disable xFormers

xFormers is not yet fully compatible with the Nightly build on Windows. This patch forces the code to use Native PyTorch Attention (SDPA) to prevent crashes.

python -c "path='src/wrapper.py'; txt=open(path, 'r', encoding='utf-8').read().replace('self.enable_xformers_memory_efficient_attention()', '# self.enable_xformers_memory_efficient_attention()'); open(path, 'w', encoding='utf-8').write(txt); print('✅ Patched src/wrapper.py')"

5. Download Weights & Run

python tools/download_weights.py
python inference_online.py

Summary of issues:
The core issue wasn't that the code was "broken," but that your setup (Windows + Brand New RTX 5080) was incompatible with the project's default instructions (Linux + Older GPUs).

Here is the breakdown of the specific conflicts:

Hardware Mismatch (The "Kernel" Error)

The Issue: Your RTX 5080 (Blackwell architecture) speaks a new language ("sm_120") that the older version of PyTorch pinned in the requirements (torch==2.1) didn't understand.

The Result: CUDA error: no kernel image available. The engine literally didn't have the machine code instructions to run on your card.

The Fix: Forcing an upgrade to the "Nightly" PyTorch build (CUDA 12.8), which includes support for your new GPU.

OS Mismatch (The "avformat" & "TensorRT" Errors)

The Issue: The requirements.txt file listed packages like av (compiled from source) and tensorrt (NVIDIA's specialized Linux accelerator). These require complex C++ build tools on Windows that you didn't have.

The Result: Compilation failures (LNK1181: cannot open input file 'avformat.lib') and crashes when the code tried to import modules that weren't there.

The Fix: Skipping the Linux-only packages and installing the pre-compiled binary version of av (via pip install av --only-binary=:all: or simply letting pip find the wheel).

Library Conflicts (The "AttributeError" & "ImportError")

The Issue:

numpy updated to version 2.0 recently, which broke older libraries like mediapipe (causing AttributeError: module 'numpy' has no attribute 'ndarray').

xformers (a speed booster) was hard-coded into the script but isn't yet fully compatible with the bleeding-edge PyTorch Nightly on Windows.

The Result: The script would crash immediately upon starting or when trying to load models.

The Fix: Downgrading numpy (<2) and patching the Python code to disable xformers so it falls back to standard (but still fast) PyTorch attention.

In short: You were trying to run a Linux-native research project on a cutting-edge Windows gaming PC. We had to replace the "engine" (PyTorch), swap out the "tires" (dependencies), and hot-wire the "ignition" (code patches) to get it running.

[huai-chang]

It is indeed super delayed, also infuse doesn't work when you want to change the ref image. It always stays on the initial image you selected.
...

@huai-chang I did pip install websockets and run again now all works perfect! the only issue i see is the realtime is very slow responsure, that happens why?

• Currently, we have only implemented a simple demo, so changing the reference image is not supported yet. I have added this to the todo list.

• Regarding Latency: Latency varies depending on your device's computing power. You can try the following methods to optimize it:

  1. Lower the "Driving FPS" setting in the WebUI to reduce the computational workload.
  2. You can increase the multiplier (e.g., set to num_frames_needed * 4 or higher) to better match your device's inference speed.

     PersonaLive/webcam/util.py

     Line 73 in 6953d1a

     read_size = min(queue.qsize(), num_frames_needed * 3)

---

The optimal state is when input_queue size stays around 4.

PersonaLive/webcam/vid2vid.py

Line 135 in 8a9be86

print("input_queue size = ", input_queue.qsize())