Open
Description
🚀 The feature, motivation and pitch
Having good basic pytorch support for inferencing LLMs is key to continued success of pytorch. Vision LLM models tend to have uneven support on mainstream inferencing engines like Llama.cpp due to the need to reimplement CLIP/SIGLIP etc. Pytorch could natively support performant vision LLMs with quantization on ARM devices, which would make a big difference in usability.
Alternatives
No response
Additional context
No response
RFC (Optional)
No response