BLIP-2 float16 example does not work

[nhatkhtn]

System Info

• transformers version: 4.50.0
• Platform: Linux-4.18.0-553.40.1.el8_10.x86_64-x86_64-with-glibc2.28
• Python version: 3.12.8
• Huggingface_hub version: 0.28.0
• Safetensors version: 0.5.2
• Accelerate version: 1.5.2
• Accelerate config: not found
• DeepSpeed version: not installed
• PyTorch version (GPU?): 2.5.1 (True)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using distributed or parallel set-up in script?: No
• Using GPU in script?: Yes
• GPU type: NVIDIA A100-SXM4-80GB

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

The following code, taken from the official documentation, does not work.

from PIL import Image
import requests
from transformers import Blip2Processor, Blip2Model
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"

processor = Blip2Processor.from_pretrained("Salesforce/blip2-opt-2.7b")
model = Blip2Model.from_pretrained("Salesforce/blip2-opt-2.7b", torch_dtype=torch.float16)
model.to(device)
url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)

prompt = "Question: how many cats are there? Answer:"
inputs = processor(images=image, text=prompt, return_tensors="pt").to(device, torch.float16)

outputs = model(**inputs)

Running the above code gives the error

RuntimeError: expected scalar type Float but found Half

Expected behavior

There should be no errors.

[zucchini-nlp]

It was fixed by #37068, we'll do a patch release for that :)

[ArthurZucker]

Patch is out: https://github.com/huggingface/transformers/releases/tag/v4.50.3

[nhatkhtn]

Hi, I don't know if it is appropriate to mention this here. I really wanted to test the patch, but the conda packages of transformers 4.50.1 and above is currently bugged. They cannot be downloaded, as seen here. Since my environment is conda-based, I cannot try the pip version.

[nhatkhtn]

Hi, I tested with transformers 4.51.3 and the issue is fixed. This issue can be closed. Kudos to the developers!