update readme file (adding usage of HuggingFace model)

[mao-code]

Based on this issue
I found that the contents of documents that describe the usage of LLaVA series model via HuggingFace are not that clear.
Therefore, I would like to add these information directly in the readme file. Hope that it can help someone who struggle with it.
If there are any mistakes, please correct me!
Thanks!

[wm-mask]

Thanks for the PR! I have a quick question, where do the models get saved? I can't find them. Thank you 🙏
Because in the documentation they ask to point to the ckpt using --model-path /path/to/model

python -m llava.serve.model_worker --host 0.0.0.0 --controller http://localhost:10000 --port 40000 --worker http://localhost:40000 --model-path /path/to/model

When I clone using git the huggingface repo under a folder checkpoints, I have:

❯ tree ./checkpoints
./checkpoints
└── llava-llama-2-13b-chat-lightning-preview
    ├── LICENSE
    ├── README.md
    ├── config.json
    ├── controller.log
    ├── generation_config.json
    ├── gradio_web_server.log
    ├── mm_projector.bin
    ├── pytorch_model-00001-of-00003.bin
    ├── pytorch_model-00002-of-00003.bin
    ├── pytorch_model-00003-of-00003.bin
    ├── pytorch_model.bin.index.json
    ├── special_tokens_map.json
    ├── tokenizer.model
    └── tokenizer_config.json

but no model is there, just bin files.

Then when I specify this folder in the --model-path in the command, the altter fails with:

2023-09-12 16:55:55 | ERROR | stderr |     return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg)
2023-09-12 16:55:55 | ERROR | stderr | RuntimeError: Internal: src/sentencepiece_processor.cc(1101) [model_proto->ParseFromArray(serialized.data(), serialized.size())]

[mao-code]

@wm-mask The model-path here I pass is the huggingface model path
Note that I use "load_from_pretrained" function to load models.

[wm-mask]

@mao-code Thanks for your reply. Sorry it's not clear.
When you say it's the hugging face model path, do you pass to it just the "liuhaotian/LLaVA-Lightning-MPT-7B-preview" name of the model? or do you pass the local path to where the model is cached by huggingface?

Also when you say "load_from_pretrained" do you mean you change some function in the code somewhere? Or just to load the model the first time for huggingface to cache it?

I followed your updated README

[mao-code]

@wm-mask Yes, the model path I used is just the hugging face model path. For the example model, it is just “liuhaotian/LLaVA-Lightning-MPT-7B-preview”.

And the load_from_pretrain is a function I import within this repository. I didn’t change anything.

[mao-code]

@wm-mask For the second part of the content that I update in the read me file, I import the eval function as I mentioned in the readme.

By doing so, you can use it to “inference” directly using the eval function.

For more detail, you can check these functions in the repository.

[haotian-liu]

Sorry for the late response. I like the idea of adding the quick start scripts, made a few mods to the language and change it the llava-v1.5-7b model in the example. Also used the collapse toggle to avoid displaying large amount of code in README. Thank you!