Hello , Please guide me with the steps involved in inferencing this model to generate csv of the chart provided #9

Activity

1. Aaryaveerkrishna23

   changed the title [-]Hello[/-] [+]Hello , Please guide me with the steps involved in inferencing this model to generate csv of the chart provided[/+] on Apr 6, 2024
2. Aaryaveerkrishna23 commented on Apr 6, 2024

   

   Aaryaveerkrishna23

   on Apr 6, 2024

   Author

   More actions

   I was facing issues like :
   1)FileNotFoundError: [Errno 2] No such file or directory: '${./base_decoder/model_base_decoder.py}/ChartVLM/base/instruction_adapter/vectorizer.pkl'
   2)ERROR: Cannot install torch==2.1.0 and triton==2.0.0 because these package versions have conflicting dependencies.
3. renqiux0302 commented on Apr 7, 2024

   

   renqiux0302

   on Apr 7, 2024

   Contributor

   More actions

   Hi @Aaryaveerkrishna23,

   1. You should modify the command (line 4 in inference.py) to the path where you save the ckpt (i.e. ../ChartVLM/base). You can download and organize the ckpt followed by Pre-trained Checkpoints of ChartVLM mentioned in README.md.

   2. It is highly recommended that you install torch ==1.13.0. You can also check the environment installation according to envs_ours.txt
4. Aaryaveerkrishna23 commented on Apr 7, 2024

   

   Aaryaveerkrishna23

   on Apr 7, 2024

   Author

   More actions

   Hi @Aaryaveerkrishna23,

   1. You should modify the command (line 4 in inference.py) to the path where you save the ckpt (i.e. ../ChartVLM/base). You can download and organize the ckpt followed by Pre-trained Checkpoints of ChartVLM mentioned in README.md.
   2. It is highly recommended that you install torch ==1.13.0. You can also check the environment installation according to envs_ours.txt

   Hello renuix I did download and organized the checkpoints as suggested by you , I got this error "IsADirectoryError: [Errno 21] Is a directory: './ChartVLM-base/auxiliary_decoder/base'
5. renqiux0302 commented on Apr 7, 2024

   

   renqiux0302

   on Apr 7, 2024

   Contributor

   More actions

   Hi @Aaryaveerkrishna23,
   There are two possible reasons:

   1. Make sure the absolute path defined is correct. (i.e. YOUR_PATH_TO/ChartVLM-base). We just provide a template of a relative path.

   2. Maybe you miss the base folder under auxiliary_decoder, please check it.
      

   Best regards
6. Aaryaveerkrishna23 commented on Apr 7, 2024

   

   Aaryaveerkrishna23

   on Apr 7, 2024

   Author

   More actions

   Hi @Aaryaveerkrishna23, There are two possible reasons:

   1. Make sure the absolute path defined is correct. (i.e. YOUR_PATH_TO/ChartVLM-base). We just provide a template of a relative path.
   2. Maybe you miss the base folder under auxiliary_decoder, please check it.
      

   Best regards

   Thanks for responding "I was getting this error , which I have pasted below also I have installed the base folder .
   my model path is """model = './ChartVLM-base' #${PATH_TO_PRETRAINED_MODEL} """"My doubt was which file exactly my model path is pointing to ?

   Traceback (most recent call last):
   File "/work/Arya/ChartVLM/ChartVLM/inference.py", line 10, in
   output = infer_ChartVLM(image, text, model)
   File "/home/jovyan/work/Arya/ChartVLM/ChartVLM/tools/ChartVLM.py", line 20, in infer_ChartVLM
   num = infer_adapter(text, model)
   File "/work/Arya/ChartVLM/ChartVLM/adapter/model_adapter.py", line 31, in infer_adapter
   vectorizer = pickle.load(file)
   _pickle.UnpicklingError: in
   valid load key, 'v'.
7. renqiux0302 commented on Apr 7, 2024

   

   renqiux0302

   on Apr 7, 2024

   Contributor

   More actions

   In inference.py, the path points to the top-level directory of ChartVLM (i.e. ./ChartVLM-base).

   We have modify the individual path to every module in adapter/model_adapter.py, base_decoder/model_base_decoder.py and auxiliary_decoder/model_auxiliary_decoder.py respectively.
8. zhanzhaoli commented on Oct 29, 2024

   

   zhanzhaoli

   on Oct 29, 2024

   More actions

   Hi @Aaryaveerkrishna23, There are two possible reasons:

   1. Make sure the absolute path defined is correct. (i.e. YOUR_PATH_TO/ChartVLM-base). We just provide a template of a relative path.
   2. Maybe you miss the base folder under auxiliary_decoder, please check it.
      

   Best regards

   Thanks for responding "I was getting this error , which I have pasted below also I have installed the base folder . my model path is """model = './ChartVLM-base' #${PATH_TO_PRETRAINED_MODEL} """"My doubt was which file exactly my model path is pointing to ?

   Traceback (most recent call last): File "/work/Arya/ChartVLM/ChartVLM/inference.py", line 10, in output = infer_ChartVLM(image, text, model) File "/home/jovyan/work/Arya/ChartVLM/ChartVLM/tools/ChartVLM.py", line 20, in infer_ChartVLM num = infer_adapter(text, model) File "/work/Arya/ChartVLM/ChartVLM/adapter/model_adapter.py", line 31, in infer_adapter vectorizer = pickle.load(file) _pickle.UnpicklingError: in valid load key, 'v'.

   I had the same issue. It turned out that git-lfs was not installed prior to downloading the pre-trained weights from Huggingface. Without git-lfs, only the pointer files were downloaded, not the actual pickle files, which caused the _pickle.UnpicklingError.

   For future reference, I got it solved by following the top answer in this post. You may also refer to this guide if you don't have sudo access to install git-lfs.