MoE Experts on AMD iGPU (Vulkan) while Layers & KV Cache on CUDA dGPU?

[Interpause]

Enlighten me if I am wrong. Would token generation speed after prompt processing increase if instead of offloading experts to CPU, experts could be offloaded to iGPU instead? (assuming there is a way to increase iGPU allocation)

[pebaryan]

IIRC, iGPU memory is just system RAM and since it is specially allocated/isolated by the OS, it has more overhead to transfer/switch the stored experts weights to the dGPU