can not reproduce the first inference step, the code released had some problem

[encounter7777]

Describe the issue

Issue:

Command:

python evaluation/inference.py --model_path 'SD-VLM-7B' --data 'MSMU' --vision_tower 'clip-vit-large-patch14-336'

Log:

 'depth_projector.patch_embedding.weight', 'depth_projector.position_embedding.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
use gt depth: False
  0%|                                                                                               | 0/2 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/workspace/SD_VLM/SD-VLM/evaluation/inference.py", line 169, in <module>
    eval_llava(args)
  File "/workspace/SD_VLM/SD-VLM/evaluation/inference.py", line 118, in eval_llava
    image_tensor = process_images([image], image_processor, model.config)[0]
  File "/workspace/SD_VLM/SD-VLM/./llava/mm_utils.py", line 171, in process_images
    image = expand2square(image, tuple(int(x*255) for x in image_processor.image_mean))
AttributeError: 'NoneType' object has no attribute 'image_mean'

Screenshots:
You may attach screenshots if it better explains the issue.

[thandal]

I had similar problems (a bunch of small broken things). I ended up rebasing off of the original LLaVA repo so that I could understand the changes that were made for SD-VLM, and got things working. See https://github.com/thandal/LLAVA-SD-VLM

[gyj155]

I had similar problems (a bunch of small broken things). I ended up rebasing off of the original LLaVA repo so that I could understand the changes that were made for SD-VLM, and got things working. See https://github.com/thandal/LLAVA-SD-VLM

Hi, thanks for sharing your fork repo. I wonder did you managed to reproduce the paper's results in your rebased repo?

[thandal]

My recollection is that the results I got were not impressive.