New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OFA-Sys Chinese-CLIP Example #3363

Open

heliang230 opened this issue Jul 25, 2024 · 1 comment

Labels

heliang230 commented Jul 25, 2024

OFA-Sys Chinese-CLIP Example

heliang230 added the enhancement label

Contributor

frankfliu commented Jul 26, 2024 •

edited

Loading

You can convert the model into onnx: https://github.com/OFA-Sys/Chinese-CLIP/blob/master/deployment.md

You will get two models, image and text model, something lie the following:

-rw-rw-r--  1 ubuntu ubuntu   5710116 Jul 26 02:37 vit-b-16.img.fp16.onnx
-rw-rw-r--  1 ubuntu ubuntu 172386816 Jul 26 02:37 vit-b-16.img.fp16.onnx.extra_file
-rw-rw-r--  1 ubuntu ubuntu   2910869 Jul 26 02:37 vit-b-16.txt.fp16.onnx
-rw-rw-r--  1 ubuntu ubuntu 816562176 Jul 26 02:37 vit-b-16.txt.fp16.onnx.extra_file

Then you can extract text feature using text model and image feature using image model.

After you get feature, you can use NDArray operators to calculate similarity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment