Add Siglip2 (naflex) support

[pySilver]

Model description

Hi! While most of the Siglip2 models are supported as they identify themselves as siglip there is a most popular model version: https://huggingface.co/google/siglip2-so400m-patch16-naflex which identifies itself as siglip2 (see https://huggingface.co/google/siglip2-so400m-patch16-naflex/blob/main/config.json).

Naflex version is a little bit different. They say:

A cherry on top is the dynamic resolution (naflex) variant. This is useful for downstream tasks sensitive to aspect ratio and resolution.

https://huggingface.co/blog/siglip2

Prerequisites

• The model is supported in Transformers (i.e., listed here)
• The model can be exported to ONNX with Optimum (i.e., listed here)

Additional information

No response

Your contribution

Unfortuantelly, I am not an expert in this field.

[xenova]

Hi there 👋 Indeed, this is a model we'd love to add support for. I'll look into it.

If anyone in the community is interested at looking at onnx export support, that would help a bunch :)

[xenova]

I got ONNX exports working, here's the first model: https://huggingface.co/onnx-community/siglip2-base-patch16-naflex-ONNX/tree/main

Next on the list is to add transformers.js code.

[pySilver]

oh @xenova you are the MAN OF THE DAY! Do you share your export code somewhere on Github? I'd love to learn from it (I could not export this model with typical tutorials & docs)

[pySilver]

On a second look - this onnx export has only vision model exported. Is that possible to have both text and image models exported? Otherwise, how one supposed to generate embeddings to text that is supposed to be checked for similarity with image. Or I am missing something?

[xenova]

Right, the export for this was a bit custom, and I aim to release the conversion script as part of our V4 release 🤞 And yes, currently I just worked on the vision export, but a similar approach should be able to be taken for the text encoder.