HunyuanImage2.1-distilled not working #9799
Description
Custom Node Testing
- I have tried disabling custom nodes and the issue persists (see how to disable custom nodes if you need help)
Expected Behavior
Distilled model working
Actual Behavior
Model loads, but outputs noise. A bunch of missing keys in console
Steps to Reproduce
Try to load https://huggingface.co/tencent/HunyuanImage-2.1/blob/main/dit/hunyuanimage2.1-distilled.safetensors in workflow from #9792
Debug Logs
unet missing: ['double_blocks.0.img_attn.qkv.weight', 'double_blocks.0.img_attn.qkv.bias', 'double_blocks.0.txt_attn.qkv.weight', 'double_blocks.0.txt_attn.qkv.bias', 'double_blocks.1.img_attn.qkv.weight', 'double_blocks.1.img_attn.qkv.bias', 'double_blocks.1.txt_attn.qkv.weight', 'double_blocks.1.txt_attn.qkv.bias', 'double_blocks.2.img_attn.qkv.weight', 'double_blocks.2.img_attn.qkv.bias', 'double_blocks.2.txt_attn.qkv.weight', 'double_blocks.2.txt_attn.qkv.bias', 'double_blocks.3.img_attn.qkv.weight', 'double_blocks.3.img_attn.qkv.bias', 'double_blocks.3.txt_attn.qkv.weight', 'double_blocks.3.txt_attn.qkv.bias', 'double_blocks.4.img_attn.qkv.weight', 'double_blocks.4.img_attn.qkv.bias', 'double_blocks.4.txt_attn.qkv.weight', 'double_blocks.4.txt_attn.qkv.bias', 'double_blocks.5.img_attn.qkv.weight', 'double_blocks.5.img_attn.qkv.bias', 'double_blocks.5.txt_attn.qkv.weight', 'double_blocks.5.txt_attn.qkv.bias', 'double_blocks.6.img_attn.qkv.weight', 'double_blocks.6.img_attn.qkv.bias', 'double_blocks.6.txt_attn.qkv.weight', 'double_blocks.6.txt_attn.qkv.bias', 'double_blocks.7.img_attn.qkv.weight', 'double_blocks.7.img_attn.qkv.bias', 'double_blocks.7.txt_attn.qkv.weight', 'double_blocks.7.txt_attn.qkv.bias', 'double_blocks.8.img_attn.qkv.weight', 'double_blocks.8.img_attn.qkv.bias', 'double_blocks.8.txt_attn.qkv.weight', 'double_blocks.8.txt_attn.qkv.bias', 'double_blocks.9.img_attn.qkv.weight', 'double_blocks.9.img_attn.qkv.bias', 'double_blocks.9.txt_attn.qkv.weight', 'double_blocks.9.txt_attn.qkv.bias', 'double_blocks.10.img_attn.qkv.weight', 'double_blocks.10.img_attn.qkv.bias', 'double_blocks.10.txt_attn.qkv.weight', 'double_blocks.10.txt_attn.qkv.bias', 'double_blocks.11.img_attn.qkv.weight', 'double_blocks.11.img_attn.qkv.bias', 'double_blocks.11.txt_attn.qkv.weight', 'double_blocks.11.txt_attn.qkv.bias', 'double_blocks.12.img_attn.qkv.weight', 'double_blocks.12.img_attn.qkv.bias', 'double_blocks.12.txt_attn.qkv.weight', 'double_blocks.12.txt_attn.qkv.bias', 'double_blocks.13.img_attn.qkv.weight', 'double_blocks.13.img_attn.qkv.bias', 'double_blocks.13.txt_attn.qkv.weight', 'double_blocks.13.txt_attn.qkv.bias', 'double_blocks.14.img_attn.qkv.weight', 'double_blocks.14.img_attn.qkv.bias', 'double_blocks.14.txt_attn.qkv.weight', 'double_blocks.14.txt_attn.qkv.bias', 'double_blocks.15.img_attn.qkv.weight', 'double_blocks.15.img_attn.qkv.bias', 'double_blocks.15.txt_attn.qkv.weight', 'double_blocks.15.txt_attn.qkv.bias', 'double_blocks.16.img_attn.qkv.weight', 'double_blocks.16.img_attn.qkv.bias', 'double_blocks.16.txt_attn.qkv.weight', 'double_blocks.16.txt_attn.qkv.bias', 'double_blocks.17.img_attn.qkv.weight', 'double_blocks.17.img_attn.qkv.bias', 'double_blocks.17.txt_attn.qkv.weight', 'double_blocks.17.txt_attn.qkv.bias', 'double_blocks.18.img_attn.qkv.weight', 'double_blocks.18.img_attn.qkv.bias', 'double_blocks.18.txt_attn.qkv.weight', 'double_blocks.18.txt_attn.qkv.bias', 'double_blocks.19.img_attn.qkv.weight', 'double_blocks.19.img_attn.qkv.bias', 'double_blocks.19.txt_attn.qkv.weight', 'double_blocks.19.txt_attn.qkv.bias', 'single_blocks.0.linear1.weight', 'single_blocks.0.linear1.bias', 'single_blocks.0.linear2.weight', 'single_blocks.0.linear2.bias', 'single_blocks.1.linear1.weight', 'single_blocks.1.linear1.bias', 'single_blocks.1.linear2.weight', 'single_blocks.1.linear2.bias', 'single_blocks.2.linear1.weight', 'single_blocks.2.linear1.bias', 'single_blocks.2.linear2.weight', 'single_blocks.2.linear2.bias', 'single_blocks.3.linear1.weight', 'single_blocks.3.linear1.bias', 'single_blocks.3.linear2.weight', 'single_blocks.3.linear2.bias', 'single_blocks.4.linear1.weight', 'single_blocks.4.linear1.bias', 'single_blocks.4.linear2.weight', 'single_blocks.4.linear2.bias', 'single_blocks.5.linear1.weight', 'single_blocks.5.linear1.bias', 'single_blocks.5.linear2.weight', 'single_blocks.5.linear2.bias', 'single_blocks.6.linear1.weight', 'single_blocks.6.linear1.bias', 'single_blocks.6.linear2.weight', 'single_blocks.6.linear2.bias', 'single_blocks.7.linear1.weight', 'single_blocks.7.linear1.bias', 'single_blocks.7.linear2.weight', 'single_blocks.7.linear2.bias', 'single_blocks.8.linear1.weight', 'single_blocks.8.linear1.bias', 'single_blocks.8.linear2.weight', 'single_blocks.8.linear2.bias', 'single_blocks.9.linear1.weight', 'single_blocks.9.linear1.bias', 'single_blocks.9.linear2.weight', 'single_blocks.9.linear2.bias', 'single_blocks.10.linear1.weight', 'single_blocks.10.linear1.bias', 'single_blocks.10.linear2.weight', 'single_blocks.10.linear2.bias', 'single_blocks.11.linear1.weight', 'single_blocks.11.linear1.bias', 'single_blocks.11.linear2.weight', 'single_blocks.11.linear2.bias', 'single_blocks.12.linear1.weight', 'single_blocks.12.linear1.bias', 'single_blocks.12.linear2.weight', 'single_blocks.12.linear2.bias', 'single_blocks.13.linear1.weight', 'single_blocks.13.linear1.bias', 'single_blocks.13.linear2.weight', 'single_blocks.13.linear2.bias', 'single_blocks.14.linear1.weight', 'single_blocks.14.linear1.bias', 'single_blocks.14.linear2.weight', 'single_blocks.14.linear2.bias', 'single_blocks.15.linear1.weight', 'single_blocks.15.linear1.bias', 'single_blocks.15.linear2.weight', 'single_blocks.15.linear2.bias', 'single_blocks.16.linear1.weight', 'single_blocks.16.linear1.bias', 'single_blocks.16.linear2.weight', 'single_blocks.16.linear2.bias', 'single_blocks.17.linear1.weight', 'single_blocks.17.linear1.bias', 'single_blocks.17.linear2.weight', 'single_blocks.17.linear2.bias', 'single_blocks.18.linear1.weight', 'single_blocks.18.linear1.bias', 'single_blocks.18.linear2.weight', 'single_blocks.18.linear2.bias', 'single_blocks.19.linear1.weight', 'single_blocks.19.linear1.bias', 'single_blocks.19.linear2.weight', 'single_blocks.19.linear2.bias', 'single_blocks.20.linear1.weight', 'single_blocks.20.linear1.bias', 'single_blocks.20.linear2.weight', 'single_blocks.20.linear2.bias', 'single_blocks.21.linear1.weight', 'single_blocks.21.linear1.bias', 'single_blocks.21.linear2.weight', 'single_blocks.21.linear2.bias', 'single_blocks.22.linear1.weight', 'single_blocks.22.linear1.bias', 'single_blocks.22.linear2.weight', 'single_blocks.22.linear2.bias', 'single_blocks.23.linear1.weight', 'single_blocks.23.linear1.bias', 'single_blocks.23.linear2.weight', 'single_blocks.23.linear2.bias', 'single_blocks.24.linear1.weight', 'single_blocks.24.linear1.bias', 'single_blocks.24.linear2.weight', 'single_blocks.24.linear2.bias', 'single_blocks.25.linear1.weight', 'single_blocks.25.linear1.bias', 'single_blocks.25.linear2.weight', 'single_blocks.25.linear2.bias', 'single_blocks.26.linear1.weight', 'single_blocks.26.linear1.bias', 'single_blocks.26.linear2.weight', 'single_blocks.26.linear2.bias', 'single_blocks.27.linear1.weight', 'single_blocks.27.linear1.bias', 'single_blocks.27.linear2.weight', 'single_blocks.27.linear2.bias', 'single_blocks.28.linear1.weight', 'single_blocks.28.linear1.bias', 'single_blocks.28.linear2.weight', 'single_blocks.28.linear2.bias', 'single_blocks.29.linear1.weight', 'single_blocks.29.linear1.bias', 'single_blocks.29.linear2.weight', 'single_blocks.29.linear2.bias', 'single_blocks.30.linear1.weight', 'single_blocks.30.linear1.bias', 'single_blocks.30.linear2.weight', 'single_blocks.30.linear2.bias', 'single_blocks.31.linear1.weight', 'single_blocks.31.linear1.bias', 'single_blocks.31.linear2.weight', 'single_blocks.31.linear2.bias', 'single_blocks.32.linear1.weight', 'single_blocks.32.linear1.bias', 'single_blocks.32.linear2.weight', 'single_blocks.32.linear2.bias', 'single_blocks.33.linear1.weight', 'single_blocks.33.linear1.bias', 'single_blocks.33.linear2.weight', 'single_blocks.33.linear2.bias', 'single_blocks.34.linear1.weight', 'single_blocks.34.linear1.bias', 'single_blocks.34.linear2.weight', 'single_blocks.34.linear2.bias', 'single_blocks.35.linear1.weight', 'single_blocks.35.linear1.bias', 'single_blocks.35.linear2.weight', 'single_blocks.35.linear2.bias', 'single_blocks.36.linear1.weight', 'single_blocks.36.linear1.bias', 'single_blocks.36.linear2.weight', 'single_blocks.36.linear2.bias', 'single_blocks.37.linear1.weight', 'single_blocks.37.linear1.bias', 'single_blocks.37.linear2.weight', 'single_blocks.37.linear2.bias', 'single_blocks.38.linear1.weight', 'single_blocks.38.linear1.bias', 'single_blocks.38.linear2.weight', 'single_blocks.38.linear2.bias', 'single_blocks.39.linear1.weight', 'single_blocks.39.linear1.bias', 'single_blocks.39.linear2.weight', 'single_blocks.39.linear2.bias']
unet unexpected: ['time_r_in.in_layer.bias', 'time_r_in.in_layer.weight', 'time_r_in.out_layer.bias', 'time_r_in.out_layer.weight', 'double_blocks.0.img_attn_k.bias', 'double_blocks.0.img_attn_k.weight', 'double_blocks.0.img_attn_q.bias', 'double_blocks.0.img_attn_q.weight', 'double_blocks.0.img_attn_v.bias', 'double_blocks.0.img_attn_v.weight', 'double_blocks.0.txt_attn_k.bias', 'double_blocks.0.txt_attn_k.weight', 'double_blocks.0.txt_attn_q.bias', 'double_blocks.0.txt_attn_q.weight', 'double_blocks.0.txt_attn_v.bias', 'double_blocks.0.txt_attn_v.weight', 'double_blocks.1.img_attn_k.bias', 'double_blocks.1.img_attn_k.weight', 'double_blocks.1.img_attn_q.bias', 'double_blocks.1.img_attn_q.weight', 'double_blocks.1.img_attn_v.bias', 'double_blocks.1.img_attn_v.weight', 'double_blocks.1.txt_attn_k.bias', 'double_blocks.1.txt_attn_k.weight', 'double_blocks.1.txt_attn_q.bias', 'double_blocks.1.txt_attn_q.weight', 'double_blocks.1.txt_attn_v.bias', 'double_blocks.1.txt_attn_v.weight', 'double_blocks.2.img_attn_k.bias', 'double_blocks.2.img_attn_k.weight', 'double_blocks.2.img_attn_q.bias', 'double_blocks.2.img_attn_q.weight', 'double_blocks.2.img_attn_v.bias', 'double_blocks.2.img_attn_v.weight', 'double_blocks.2.txt_attn_k.bias', 'double_blocks.2.txt_attn_k.weight', 'double_blocks.2.txt_attn_q.bias', 'double_blocks.2.txt_attn_q.weight', 'double_blocks.2.txt_attn_v.bias', 'double_blocks.2.txt_attn_v.weight', 'double_blocks.3.img_attn_k.bias', 'double_blocks.3.img_attn_k.weight', 'double_blocks.3.img_attn_q.bias', 'double_blocks.3.img_attn_q.weight', 'double_blocks.3.img_attn_v.bias', 'double_blocks.3.img_attn_v.weight', 'double_blocks.3.txt_attn_k.bias', 'double_blocks.3.txt_attn_k.weight', 'double_blocks.3.txt_attn_q.bias', 'double_blocks.3.txt_attn_q.weight', 'double_blocks.3.txt_attn_v.bias', 'double_blocks.3.txt_attn_v.weight', 'double_blocks.4.img_attn_k.bias', 'double_blocks.4.img_attn_k.weight', 'double_blocks.4.img_attn_q.bias', 'double_blocks.4.img_attn_q.weight', 'double_blocks.4.img_attn_v.bias', 'double_blocks.4.img_attn_v.weight', 'double_blocks.4.txt_attn_k.bias', 'double_blocks.4.txt_attn_k.weight', 'double_blocks.4.txt_attn_q.bias', 'double_blocks.4.txt_attn_q.weight', 'double_blocks.4.txt_attn_v.bias', 'double_blocks.4.txt_attn_v.weight', 'double_blocks.5.img_attn_k.bias', 'double_blocks.5.img_attn_k.weight', 'double_blocks.5.img_attn_q.bias', 'double_blocks.5.img_attn_q.weight', 'double_blocks.5.img_attn_v.bias', 'double_blocks.5.img_attn_v.weight', 'double_blocks.5.txt_attn_k.bias', 'double_blocks.5.txt_attn_k.weight', 'double_blocks.5.txt_attn_q.bias', 'double_blocks.5.txt_attn_q.weight', 'double_blocks.5.txt_attn_v.bias', 'double_blocks.5.txt_attn_v.weight', 'double_blocks.6.img_attn_k.bias', 'double_blocks.6.img_attn_k.weight', 'double_blocks.6.img_attn_q.bias', 'double_blocks.6.img_attn_q.weight', 'double_blocks.6.img_attn_v.bias', 'double_blocks.6.img_attn_v.weight', 'double_blocks.6.txt_attn_k.bias', 'double_blocks.6.txt_attn_k.weight', 'double_blocks.6.txt_attn_q.bias', 'double_blocks.6.txt_attn_q.weight', 'double_blocks.6.txt_attn_v.bias', 'double_blocks.6.txt_attn_v.weight', 'double_blocks.7.img_attn_k.bias', 'double_blocks.7.img_attn_k.weight', 'double_blocks.7.img_attn_q.bias', 'double_blocks.7.img_attn_q.weight', 'double_blocks.7.img_attn_v.bias', 'double_blocks.7.img_attn_v.weight', 'double_blocks.7.txt_attn_k.bias', 'double_blocks.7.txt_attn_k.weight', 'double_blocks.7.txt_attn_q.bias', 'double_blocks.7.txt_attn_q.weight', 'double_blocks.7.txt_attn_v.bias', 'double_blocks.7.txt_attn_v.weight', 'double_blocks.8.img_attn_k.bias', 'double_blocks.8.img_attn_k.weight', 'double_blocks.8.img_attn_q.bias', 'double_blocks.8.img_attn_q.weight', 'double_blocks.8.img_attn_v.bias', 'double_blocks.8.img_attn_v.weight', 'double_blocks.8.txt_attn_k.bias', 'double_blocks.8.txt_attn_k.weight', 'double_blocks.8.txt_attn_q.bias', 'double_blocks.8.txt_attn_q.weight', 'double_blocks.8.txt_attn_v.bias', 'double_blocks.8.txt_attn_v.weight', 'double_blocks.9.img_attn_k.bias', 'double_blocks.9.img_attn_k.weight', 'double_blocks.9.img_attn_q.bias', 'double_blocks.9.img_attn_q.weight', 'double_blocks.9.img_attn_v.bias', 'double_blocks.9.img_attn_v.weight', 'double_blocks.9.txt_attn_k.bias', 'double_blocks.9.txt_attn_k.weight', 'double_blocks.9.txt_attn_q.bias', 'double_blocks.9.txt_attn_q.weight', 'double_blocks.9.txt_attn_v.bias', 'double_blocks.9.txt_attn_v.weight', 'double_blocks.10.img_attn_k.bias', 'double_blocks.10.img_attn_k.weight', 'double_blocks.10.img_attn_q.bias', 'double_blocks.10.img_attn_q.weight', 'double_blocks.10.img_attn_v.bias', 'double_blocks.10.img_attn_v.weight', 'double_blocks.10.txt_attn_k.bias', 'double_blocks.10.txt_attn_k.weight', 'double_blocks.10.txt_attn_q.bias', 'double_blocks.10.txt_attn_q.weight', 'double_blocks.10.txt_attn_v.bias', 'double_blocks.10.txt_attn_v.weight', 'double_blocks.11.img_attn_k.bias', 'double_blocks.11.img_attn_k.weight', 'double_blocks.11.img_attn_q.bias', 'double_blocks.11.img_attn_q.weight', 'double_blocks.11.img_attn_v.bias', 'double_blocks.11.img_attn_v.weight', 'double_blocks.11.txt_attn_k.bias', 'double_blocks.11.txt_attn_k.weight', 'double_blocks.11.txt_attn_q.bias', 'double_blocks.11.txt_attn_q.weight', 'double_blocks.11.txt_attn_v.bias', 'double_blocks.11.txt_attn_v.weight', 'double_blocks.12.img_attn_k.bias', 'double_blocks.12.img_attn_k.weight', 'double_blocks.12.img_attn_q.bias', 'double_blocks.12.img_attn_q.weight', 'double_blocks.12.img_attn_v.bias', 'double_blocks.12.img_attn_v.weight', 'double_blocks.12.txt_attn_k.bias', 'double_blocks.12.txt_attn_k.weight', 'double_blocks.12.txt_attn_q.bias', 'double_blocks.12.txt_attn_q.weight', 'double_blocks.12.txt_attn_v.bias', 'double_blocks.12.txt_attn_v.weight', 'double_blocks.13.img_attn_k.bias', 'double_blocks.13.img_attn_k.weight', 'double_blocks.13.img_attn_q.bias', 'double_blocks.13.img_attn_q.weight', 'double_blocks.13.img_attn_v.bias', 'double_blocks.13.img_attn_v.weight', 'double_blocks.13.txt_attn_k.bias', 'double_blocks.13.txt_attn_k.weight', 'double_blocks.13.txt_attn_q.bias', 'double_blocks.13.txt_attn_q.weight', 'double_blocks.13.txt_attn_v.bias', 'double_blocks.13.txt_attn_v.weight', 'double_blocks.14.img_attn_k.bias', 'double_blocks.14.img_attn_k.weight', 'double_blocks.14.img_attn_q.bias', 'double_blocks.14.img_attn_q.weight', 'double_blocks.14.img_attn_v.bias', 'double_blocks.14.img_attn_v.weight', 'double_blocks.14.txt_attn_k.bias', 'double_blocks.14.txt_attn_k.weight', 'double_blocks.14.txt_attn_q.bias', 'double_blocks.14.txt_attn_q.weight', 'double_blocks.14.txt_attn_v.bias', 'double_blocks.14.txt_attn_v.weight', 'double_blocks.15.img_attn_k.bias', 'double_blocks.15.img_attn_k.weight', 'double_blocks.15.img_attn_q.bias', 'double_blocks.15.img_attn_q.weight', 'double_blocks.15.img_attn_v.bias', 'double_blocks.15.img_attn_v.weight', 'double_blocks.15.txt_attn_k.bias', 'double_blocks.15.txt_attn_k.weight', 'double_blocks.15.txt_attn_q.bias', 'double_blocks.15.txt_attn_q.weight', 'double_blocks.15.txt_attn_v.bias', 'double_blocks.15.txt_attn_v.weight', 'double_blocks.16.img_attn_k.bias', 'double_blocks.16.img_attn_k.weight', 'double_blocks.16.img_attn_q.bias', 'double_blocks.16.img_attn_q.weight', 'double_blocks.16.img_attn_v.bias', 'double_blocks.16.img_attn_v.weight', 'double_blocks.16.txt_attn_k.bias', 'double_blocks.16.txt_attn_k.weight', 'double_blocks.16.txt_attn_q.bias', 'double_blocks.16.txt_attn_q.weight', 'double_blocks.16.txt_attn_v.bias', 'double_blocks.16.txt_attn_v.weight', 'double_blocks.17.img_attn_k.bias', 'double_blocks.17.img_attn_k.weight', 'double_blocks.17.img_attn_q.bias', 'double_blocks.17.img_attn_q.weight', 'double_blocks.17.img_attn_v.bias', 'double_blocks.17.img_attn_v.weight', 'double_blocks.17.txt_attn_k.bias', 'double_blocks.17.txt_attn_k.weight', 'double_blocks.17.txt_attn_q.bias', 'double_blocks.17.txt_attn_q.weight', 'double_blocks.17.txt_attn_v.bias', 'double_blocks.17.txt_attn_v.weight', 'double_blocks.18.img_attn_k.bias', 'double_blocks.18.img_attn_k.weight', 'double_blocks.18.img_attn_q.bias', 'double_blocks.18.img_attn_q.weight', 'double_blocks.18.img_attn_v.bias', 'double_blocks.18.img_attn_v.weight', 'double_blocks.18.txt_attn_k.bias', 'double_blocks.18.txt_attn_k.weight', 'double_blocks.18.txt_attn_q.bias', 'double_blocks.18.txt_attn_q.weight', 'double_blocks.18.txt_attn_v.bias', 'double_blocks.18.txt_attn_v.weight', 'double_blocks.19.img_attn_k.bias', 'double_blocks.19.img_attn_k.weight', 'double_blocks.19.img_attn_q.bias', 'double_blocks.19.img_attn_q.weight', 'double_blocks.19.img_attn_v.bias', 'double_blocks.19.img_attn_v.weight', 'double_blocks.19.txt_attn_k.bias', 'double_blocks.19.txt_attn_k.weight', 'double_blocks.19.txt_attn_q.bias', 'double_blocks.19.txt_attn_q.weight', 'double_blocks.19.txt_attn_v.bias', 'double_blocks.19.txt_attn_v.weight', 'single_blocks.0.linear1_k.bias', 'single_blocks.0.linear1_k.weight', 'single_blocks.0.linear1_mlp.bias', 'single_blocks.0.linear1_mlp.weight', 'single_blocks.0.linear1_q.bias', 'single_blocks.0.linear1_q.weight', 'single_blocks.0.linear1_v.bias', 'single_blocks.0.linear1_v.weight', 'single_blocks.0.linear2.fc.bias', 'single_blocks.0.linear2.fc.weight', 'single_blocks.1.linear1_k.bias', 'single_blocks.1.linear1_k.weight', 'single_blocks.1.linear1_mlp.bias', 'single_blocks.1.linear1_mlp.weight', 'single_blocks.1.linear1_q.bias', 'single_blocks.1.linear1_q.weight', 'single_blocks.1.linear1_v.bias', 'single_blocks.1.linear1_v.weight', 'single_blocks.1.linear2.fc.bias', 'single_blocks.1.linear2.fc.weight', 'single_blocks.2.linear1_k.bias', 'single_blocks.2.linear1_k.weight', 'single_blocks.2.linear1_mlp.bias', 'single_blocks.2.linear1_mlp.weight', 'single_blocks.2.linear1_q.bias', 'single_blocks.2.linear1_q.weight', 'single_blocks.2.linear1_v.bias', 'single_blocks.2.linear1_v.weight', 'single_blocks.2.linear2.fc.bias', 'single_blocks.2.linear2.fc.weight', 'single_blocks.3.linear1_k.bias', 'single_blocks.3.linear1_k.weight', 'single_blocks.3.linear1_mlp.bias', 'single_blocks.3.linear1_mlp.weight', 'single_blocks.3.linear1_q.bias', 'single_blocks.3.linear1_q.weight', 'single_blocks.3.linear1_v.bias', 'single_blocks.3.linear1_v.weight', 'single_blocks.3.linear2.fc.bias', 'single_blocks.3.linear2.fc.weight', 'single_blocks.4.linear1_k.bias', 'single_blocks.4.linear1_k.weight', 'single_blocks.4.linear1_mlp.bias', 'single_blocks.4.linear1_mlp.weight', 'single_blocks.4.linear1_q.bias', 'single_blocks.4.linear1_q.weight', 'single_blocks.4.linear1_v.bias', 'single_blocks.4.linear1_v.weight', 'single_blocks.4.linear2.fc.bias', 'single_blocks.4.linear2.fc.weight', 'single_blocks.5.linear1_k.bias', 'single_blocks.5.linear1_k.weight', 'single_blocks.5.linear1_mlp.bias', 'single_blocks.5.linear1_mlp.weight', 'single_blocks.5.linear1_q.bias', 'single_blocks.5.linear1_q.weight', 'single_blocks.5.linear1_v.bias', 'single_blocks.5.linear1_v.weight', 'single_blocks.5.linear2.fc.bias', 'single_blocks.5.linear2.fc.weight', 'single_blocks.6.linear1_k.bias', 'single_blocks.6.linear1_k.weight', 'single_blocks.6.linear1_mlp.bias', 'single_blocks.6.linear1_mlp.weight', 'single_blocks.6.linear1_q.bias', 'single_blocks.6.linear1_q.weight', 'single_blocks.6.linear1_v.bias', 'single_blocks.6.linear1_v.weight', 'single_blocks.6.linear2.fc.bias', 'single_blocks.6.linear2.fc.weight', 'single_blocks.7.linear1_k.bias', 'single_blocks.7.linear1_k.weight', 'single_blocks.7.linear1_mlp.bias', 'single_blocks.7.linear1_mlp.weight', 'single_blocks.7.linear1_q.bias', 'single_blocks.7.linear1_q.weight', 'single_blocks.7.linear1_v.bias', 'single_blocks.7.linear1_v.weight', 'single_blocks.7.linear2.fc.bias', 'single_blocks.7.linear2.fc.weight', 'single_blocks.8.linear1_k.bias', 'single_blocks.8.linear1_k.weight', 'single_blocks.8.linear1_mlp.bias', 'single_blocks.8.linear1_mlp.weight', 'single_blocks.8.linear1_q.bias', 'single_blocks.8.linear1_q.weight', 'single_blocks.8.linear1_v.bias', 'single_blocks.8.linear1_v.weight', 'single_blocks.8.linear2.fc.bias', 'single_blocks.8.linear2.fc.weight', 'single_blocks.9.linear1_k.bias', 'single_blocks.9.linear1_k.weight', 'single_blocks.9.linear1_mlp.bias', 'single_blocks.9.linear1_mlp.weight', 'single_blocks.9.linear1_q.bias', 'single_blocks.9.linear1_q.weight', 'single_blocks.9.linear1_v.bias', 'single_blocks.9.linear1_v.weight', 'single_blocks.9.linear2.fc.bias', 'single_blocks.9.linear2.fc.weight', 'single_blocks.10.linear1_k.bias', 'single_blocks.10.linear1_k.weight', 'single_blocks.10.linear1_mlp.bias', 'single_blocks.10.linear1_mlp.weight', 'single_blocks.10.linear1_q.bias', 'single_blocks.10.linear1_q.weight', 'single_blocks.10.linear1_v.bias', 'single_blocks.10.linear1_v.weight', 'single_blocks.10.linear2.fc.bias', 'single_blocks.10.linear2.fc.weight', 'single_blocks.11.linear1_k.bias', 'single_blocks.11.linear1_k.weight', 'single_blocks.11.linear1_mlp.bias', 'single_blocks.11.linear1_mlp.weight', 'single_blocks.11.linear1_q.bias', 'single_blocks.11.linear1_q.weight', 'single_blocks.11.linear1_v.bias', 'single_blocks.11.linear1_v.weight', 'single_blocks.11.linear2.fc.bias', 'single_blocks.11.linear2.fc.weight', 'single_blocks.12.linear1_k.bias', 'single_blocks.12.linear1_k.weight', 'single_blocks.12.linear1_mlp.bias', 'single_blocks.12.linear1_mlp.weight', 'single_blocks.12.linear1_q.bias', 'single_blocks.12.linear1_q.weight', 'single_blocks.12.linear1_v.bias', 'single_blocks.12.linear1_v.weight', 'single_blocks.12.linear2.fc.bias', 'single_blocks.12.linear2.fc.weight', 'single_blocks.13.linear1_k.bias', 'single_blocks.13.linear1_k.weight', 'single_blocks.13.linear1_mlp.bias', 'single_blocks.13.linear1_mlp.weight', 'single_blocks.13.linear1_q.bias', 'single_blocks.13.linear1_q.weight', 'single_blocks.13.linear1_v.bias', 'single_blocks.13.linear1_v.weight', 'single_blocks.13.linear2.fc.bias', 'single_blocks.13.linear2.fc.weight', 'single_blocks.14.linear1_k.bias', 'single_blocks.14.linear1_k.weight', 'single_blocks.14.linear1_mlp.bias', 'single_blocks.14.linear1_mlp.weight', 'single_blocks.14.linear1_q.bias', 'single_blocks.14.linear1_q.weight', 'single_blocks.14.linear1_v.bias', 'single_blocks.14.linear1_v.weight', 'single_blocks.14.linear2.fc.bias', 'single_blocks.14.linear2.fc.weight', 'single_blocks.15.linear1_k.bias', 'single_blocks.15.linear1_k.weight', 'single_blocks.15.linear1_mlp.bias', 'single_blocks.15.linear1_mlp.weight', 'single_blocks.15.linear1_q.bias', 'single_blocks.15.linear1_q.weight', 'single_blocks.15.linear1_v.bias', 'single_blocks.15.linear1_v.weight', 'single_blocks.15.linear2.fc.bias', 'single_blocks.15.linear2.fc.weight', 'single_blocks.16.linear1_k.bias', 'single_blocks.16.linear1_k.weight', 'single_blocks.16.linear1_mlp.bias', 'single_blocks.16.linear1_mlp.weight', 'single_blocks.16.linear1_q.bias', 'single_blocks.16.linear1_q.weight', 'single_blocks.16.linear1_v.bias', 'single_blocks.16.linear1_v.weight', 'single_blocks.16.linear2.fc.bias', 'single_blocks.16.linear2.fc.weight', 'single_blocks.17.linear1_k.bias', 'single_blocks.17.linear1_k.weight', 'single_blocks.17.linear1_mlp.bias', 'single_blocks.17.linear1_mlp.weight', 'single_blocks.17.linear1_q.bias', 'single_blocks.17.linear1_q.weight', 'single_blocks.17.linear1_v.bias', 'single_blocks.17.linear1_v.weight', 'single_blocks.17.linear2.fc.bias', 'single_blocks.17.linear2.fc.weight', 'single_blocks.18.linear1_k.bias', 'single_blocks.18.linear1_k.weight', 'single_blocks.18.linear1_mlp.bias', 'single_blocks.18.linear1_mlp.weight', 'single_blocks.18.linear1_q.bias', 'single_blocks.18.linear1_q.weight', 'single_blocks.18.linear1_v.bias', 'single_blocks.18.linear1_v.weight', 'single_blocks.18.linear2.fc.bias', 'single_blocks.18.linear2.fc.weight', 'single_blocks.19.linear1_k.bias', 'single_blocks.19.linear1_k.weight', 'single_blocks.19.linear1_mlp.bias', 'single_blocks.19.linear1_mlp.weight', 'single_blocks.19.linear1_q.bias', 'single_blocks.19.linear1_q.weight', 'single_blocks.19.linear1_v.bias', 'single_blocks.19.linear1_v.weight', 'single_blocks.19.linear2.fc.bias', 'single_blocks.19.linear2.fc.weight', 'single_blocks.20.linear1_k.bias', 'single_blocks.20.linear1_k.weight', 'single_blocks.20.linear1_mlp.bias', 'single_blocks.20.linear1_mlp.weight', 'single_blocks.20.linear1_q.bias', 'single_blocks.20.linear1_q.weight', 'single_blocks.20.linear1_v.bias', 'single_blocks.20.linear1_v.weight', 'single_blocks.20.linear2.fc.bias', 'single_blocks.20.linear2.fc.weight', 'single_blocks.21.linear1_k.bias', 'single_blocks.21.linear1_k.weight', 'single_blocks.21.linear1_mlp.bias', 'single_blocks.21.linear1_mlp.weight', 'single_blocks.21.linear1_q.bias', 'single_blocks.21.linear1_q.weight', 'single_blocks.21.linear1_v.bias', 'single_blocks.21.linear1_v.weight', 'single_blocks.21.linear2.fc.bias', 'single_blocks.21.linear2.fc.weight', 'single_blocks.22.linear1_k.bias', 'single_blocks.22.linear1_k.weight', 'single_blocks.22.linear1_mlp.bias', 'single_blocks.22.linear1_mlp.weight', 'single_blocks.22.linear1_q.bias', 'single_blocks.22.linear1_q.weight', 'single_blocks.22.linear1_v.bias', 'single_blocks.22.linear1_v.weight', 'single_blocks.22.linear2.fc.bias', 'single_blocks.22.linear2.fc.weight', 'single_blocks.23.linear1_k.bias', 'single_blocks.23.linear1_k.weight', 'single_blocks.23.linear1_mlp.bias', 'single_blocks.23.linear1_mlp.weight', 'single_blocks.23.linear1_q.bias', 'single_blocks.23.linear1_q.weight', 'single_blocks.23.linear1_v.bias', 'single_blocks.23.linear1_v.weight', 'single_blocks.23.linear2.fc.bias', 'single_blocks.23.linear2.fc.weight', 'single_blocks.24.linear1_k.bias', 'single_blocks.24.linear1_k.weight', 'single_blocks.24.linear1_mlp.bias', 'single_blocks.24.linear1_mlp.weight', 'single_blocks.24.linear1_q.bias', 'single_blocks.24.linear1_q.weight', 'single_blocks.24.linear1_v.bias', 'single_blocks.24.linear1_v.weight', 'single_blocks.24.linear2.fc.bias', 'single_blocks.24.linear2.fc.weight', 'single_blocks.25.linear1_k.bias', 'single_blocks.25.linear1_k.weight', 'single_blocks.25.linear1_mlp.bias', 'single_blocks.25.linear1_mlp.weight', 'single_blocks.25.linear1_q.bias', 'single_blocks.25.linear1_q.weight', 'single_blocks.25.linear1_v.bias', 'single_blocks.25.linear1_v.weight', 'single_blocks.25.linear2.fc.bias', 'single_blocks.25.linear2.fc.weight', 'single_blocks.26.linear1_k.bias', 'single_blocks.26.linear1_k.weight', 'single_blocks.26.linear1_mlp.bias', 'single_blocks.26.linear1_mlp.weight', 'single_blocks.26.linear1_q.bias', 'single_blocks.26.linear1_q.weight', 'single_blocks.26.linear1_v.bias', 'single_blocks.26.linear1_v.weight', 'single_blocks.26.linear2.fc.bias', 'single_blocks.26.linear2.fc.weight', 'single_blocks.27.linear1_k.bias', 'single_blocks.27.linear1_k.weight', 'single_blocks.27.linear1_mlp.bias', 'single_blocks.27.linear1_mlp.weight', 'single_blocks.27.linear1_q.bias', 'single_blocks.27.linear1_q.weight', 'single_blocks.27.linear1_v.bias', 'single_blocks.27.linear1_v.weight', 'single_blocks.27.linear2.fc.bias', 'single_blocks.27.linear2.fc.weight', 'single_blocks.28.linear1_k.bias', 'single_blocks.28.linear1_k.weight', 'single_blocks.28.linear1_mlp.bias', 'single_blocks.28.linear1_mlp.weight', 'single_blocks.28.linear1_q.bias', 'single_blocks.28.linear1_q.weight', 'single_blocks.28.linear1_v.bias', 'single_blocks.28.linear1_v.weight', 'single_blocks.28.linear2.fc.bias', 'single_blocks.28.linear2.fc.weight', 'single_blocks.29.linear1_k.bias', 'single_blocks.29.linear1_k.weight', 'single_blocks.29.linear1_mlp.bias', 'single_blocks.29.linear1_mlp.weight', 'single_blocks.29.linear1_q.bias', 'single_blocks.29.linear1_q.weight', 'single_blocks.29.linear1_v.bias', 'single_blocks.29.linear1_v.weight', 'single_blocks.29.linear2.fc.bias', 'single_blocks.29.linear2.fc.weight', 'single_blocks.30.linear1_k.bias', 'single_blocks.30.linear1_k.weight', 'single_blocks.30.linear1_mlp.bias', 'single_blocks.30.linear1_mlp.weight', 'single_blocks.30.linear1_q.bias', 'single_blocks.30.linear1_q.weight', 'single_blocks.30.linear1_v.bias', 'single_blocks.30.linear1_v.weight', 'single_blocks.30.linear2.fc.bias', 'single_blocks.30.linear2.fc.weight', 'single_blocks.31.linear1_k.bias', 'single_blocks.31.linear1_k.weight', 'single_blocks.31.linear1_mlp.bias', 'single_blocks.31.linear1_mlp.weight', 'single_blocks.31.linear1_q.bias', 'single_blocks.31.linear1_q.weight', 'single_blocks.31.linear1_v.bias', 'single_blocks.31.linear1_v.weight', 'single_blocks.31.linear2.fc.bias', 'single_blocks.31.linear2.fc.weight', 'single_blocks.32.linear1_k.bias', 'single_blocks.32.linear1_k.weight', 'single_blocks.32.linear1_mlp.bias', 'single_blocks.32.linear1_mlp.weight', 'single_blocks.32.linear1_q.bias', 'single_blocks.32.linear1_q.weight', 'single_blocks.32.linear1_v.bias', 'single_blocks.32.linear1_v.weight', 'single_blocks.32.linear2.fc.bias', 'single_blocks.32.linear2.fc.weight', 'single_blocks.33.linear1_k.bias', 'single_blocks.33.linear1_k.weight', 'single_blocks.33.linear1_mlp.bias', 'single_blocks.33.linear1_mlp.weight', 'single_blocks.33.linear1_q.bias', 'single_blocks.33.linear1_q.weight', 'single_blocks.33.linear1_v.bias', 'single_blocks.33.linear1_v.weight', 'single_blocks.33.linear2.fc.bias', 'single_blocks.33.linear2.fc.weight', 'single_blocks.34.linear1_k.bias', 'single_blocks.34.linear1_k.weight', 'single_blocks.34.linear1_mlp.bias', 'single_blocks.34.linear1_mlp.weight', 'single_blocks.34.linear1_q.bias', 'single_blocks.34.linear1_q.weight', 'single_blocks.34.linear1_v.bias', 'single_blocks.34.linear1_v.weight', 'single_blocks.34.linear2.fc.bias', 'single_blocks.34.linear2.fc.weight', 'single_blocks.35.linear1_k.bias', 'single_blocks.35.linear1_k.weight', 'single_blocks.35.linear1_mlp.bias', 'single_blocks.35.linear1_mlp.weight', 'single_blocks.35.linear1_q.bias', 'single_blocks.35.linear1_q.weight', 'single_blocks.35.linear1_v.bias', 'single_blocks.35.linear1_v.weight', 'single_blocks.35.linear2.fc.bias', 'single_blocks.35.linear2.fc.weight', 'single_blocks.36.linear1_k.bias', 'single_blocks.36.linear1_k.weight', 'single_blocks.36.linear1_mlp.bias', 'single_blocks.36.linear1_mlp.weight', 'single_blocks.36.linear1_q.bias', 'single_blocks.36.linear1_q.weight', 'single_blocks.36.linear1_v.bias', 'single_blocks.36.linear1_v.weight', 'single_blocks.36.linear2.fc.bias', 'single_blocks.36.linear2.fc.weight', 'single_blocks.37.linear1_k.bias', 'single_blocks.37.linear1_k.weight', 'single_blocks.37.linear1_mlp.bias', 'single_blocks.37.linear1_mlp.weight', 'single_blocks.37.linear1_q.bias', 'single_blocks.37.linear1_q.weight', 'single_blocks.37.linear1_v.bias', 'single_blocks.37.linear1_v.weight', 'single_blocks.37.linear2.fc.bias', 'single_blocks.37.linear2.fc.weight', 'single_blocks.38.linear1_k.bias', 'single_blocks.38.linear1_k.weight', 'single_blocks.38.linear1_mlp.bias', 'single_blocks.38.linear1_mlp.weight', 'single_blocks.38.linear1_q.bias', 'single_blocks.38.linear1_q.weight', 'single_blocks.38.linear1_v.bias', 'single_blocks.38.linear1_v.weight', 'single_blocks.38.linear2.fc.bias', 'single_blocks.38.linear2.fc.weight', 'single_blocks.39.linear1_k.bias', 'single_blocks.39.linear1_k.weight', 'single_blocks.39.linear1_mlp.bias', 'single_blocks.39.linear1_mlp.weight', 'single_blocks.39.linear1_q.bias', 'single_blocks.39.linear1_q.weight', 'single_blocks.39.linear1_v.bias', 'single_blocks.39.linear1_v.weight', 'single_blocks.39.linear2.fc.bias', 'single_blocks.39.linear2.fc.weight']Other
Looking at the keys in hugginface preview, distilled model has additional keys not present in non distilled version