[server] Add model argument to server cli#1584
Conversation
|
is this a breaking change if we were using |
|
@mgoin yes. If we make We could add a separate model path entry point but click can't support both AFAIK |
@dsikka per @markurtz lets add a model_path kwarg back in (can rename the positional arg to something else) and allow it to override the positional if given. (Would need to make the positional arg optional in this case I guess) |
model argument to server cli
* update model path to be an argument; remove unused openai command pathway * add model path arg and option
* update model path to be an argument; remove unused openai command pathway * add model path arg and option
Summary:
clickset-up, to get the use case described in this ticket (shown below): https://app.asana.com/0/1206109050183159/1206524727025314/fdeepsparse.server \ "zoo:llama2-7b-ultrachat200k_llama2_pretrain-pruned50_quantized" \ --integration openaiPR Update
This PR allows us to get the following cli command:
deepsparse.server --integration openai "hf:mgoin/TinyStories-1M-ds"We can run the following commands with the current set-up:
Caveats (@bfineran):
Shoutout to @rahul-tuli for his click knowledge and help