Feature request

Now that the context window is variable (per #1668) it would be helpful to have models2.json updated to populate the n_ctx field along with the correct system and user prompts.

This could be accomplished by adding in the filed contextLength or similar for each model.

Motivation

At the moment, the default context window remains 2048 tokens, albeit user-configurable. Populating the n_ctx value upon the installation of a model will allow for more effective use of this new feature.

Your contribution

Looking through the models in models2.json, I think that the correct context values are:

1. Mistral OpenOrca: 4096 (https://huggingface.co/Open-Orca/Mistral-7B-OpenOrca/blob/main/config.json "sliding_window")
2. Mistral Instruct: 4096 (https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1/blob/main/config.json "sliding_window")
3. GPT4All Falcoln: 2048 (https://huggingface.co/tiiuae/falcon-7b, the source model)
4. Orca2 Medium: ??4096 (https://arxiv.org/pdf/2311.11045.pdf p8)
5. Orca2 Full: ?4096 (https://huggingface.co/microsoft/Orca-2-13b model_max_length=4096)
6. Wizard v1.2: 4096 (https://conclusionlab.com/llm/WizardLM-WizardLM-13B-V1.2)
7. Hermes: 4096 (https://huggingface.co/NousResearch/Nous-Hermes-Llama2-13b/discussions/7)
8. GPT4All Snoozy: 2048 (https://llm.extractum.io/model/TheBloke%2FGPT4All-13B-snoozy-GPTQ,4S2tA74MFGmT3FueEg9sUt)
9. MPT Chat: 4096 (https://huggingface.co/mosaicml/mpt-7b-chat config.max_seq_len)
10. Orca Mini: ??1024 (https://huggingface.co/pankajmathur/orca_mini_3b but this is unclear)
11. EM German Mistral: 4096 (https://huggingface.co/jphme/em_german_mistral_v01/discussions/2)