Mistral-7B-instruct-v0.1 compatibility with main.py

[nvidal01]

Hi,
I managed to install mistral-7b-v.01 on a server and run the main.py script as recommended, and it works well. I wanted to test the model's abilities in chat completion, so I downloaded Mistral-7b-instruct-v0.1. But when running the same commands as for mistral-7b-v0.1, the main.py program does not work (see error below). More specifically the model.py script included in the mistral folder does not seem compatible with Mistral-7b-instruct-v0.1.
Do you know how to resolve this problem?
Thank you

` > python -m main interactive Mistral-7B-instruct-v0.1/

Traceback (most recent call last):
File "/home1/USERS/PSY-DEV/brunet/anaconda3/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home1/USERS/PSY-DEV/brunet/anaconda3/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home1/USERS/PSY-DEV/brunet/llama/mistral-src/main.py", line 142, in
fire.Fire({
File "/home1/USERS/PSY-DEV/brunet/anaconda3/lib/python3.10/site-packages/fire/core.py", line 141, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/home1/USERS/PSY-DEV/brunet/anaconda3/lib/python3.10/site-packages/fire/core.py", line 475, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/home1/USERS/PSY-DEV/brunet/anaconda3/lib/python3.10/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "/home1/USERS/PSY-DEV/brunet/llama/mistral-src/main.py", line 106, in interactive
transformer = Transformer.from_folder(Path(model_path), max_batch_size=3)
File "/home1/USERS/PSY-DEV/brunet/llama/mistral-src/mistral/model.py", line 218, in from_folder
model_args = ModelArgs(**json.loads(f.read()))
TypeError: ModelArgs.init() got an unexpected keyword argument 'use_biases'`

[JohnTailor]

Fix: Go to Mistral folder (Mistral-7B-instruct-v0.1), and change contents of "params.json" to

{"dim": 4096, "n_layers": 32, "head_dim": 128, "hidden_dim": 14336, "n_heads": 32, "n_kv_heads": 8, "norm_eps": 1e-05, "vocab_size": 32000, "sliding_window":3}

This lets you run the test, BUT: I don't get reasonable responses :-(

[eeric]

wget https://files.mistral-7b-v0-1.mistral.ai/mistral-7B-v0.1.tar
tar -xf mistral-7B-v0.1.tar

it works well.

[dagostinob]

I am experiencing the same problem as @nvidal01 - I can get Mistral-7B-v0.1 to run and generate sensible text. But the instruct version of the model (Mistral-7B-Instruct-v0.1) does not run, due to unexpected keywords in the params.json config file. If I remove keywords from params.json until I stop getting TypeErrors, as @JohnTailor did, I can get Mistral-7B-Instruct-v0.1 to run. But results are nonsensical. It would be very helpful to have a working example of the Instruct model.

[timlacroix]

Ah sorry about that !
We'll update the examples and fix that stray param ...