Is there any way to adjust parameters like -c, -b, --mlock, --min-p, --top-p, --top-k, --repeat-penalty, --no-penalize-nl and --temp etc. in api_like_OAI.py

[SeriousHulk]

          Is there any way to adjust parameters like -c, -b, --mlock, --min-p, --top-p, --top-k, --repeat-penalty, --no-penalize-nl and --temp etc. in api_like_OAI.py

Originally posted by @SeriousHulk in #4367 (comment)

[ggerganov]

Slightly off-topic, but what does api_like_OAI.py currently offer that server does not?

I was thinking of removing that script since I believe server already support the OAI API

[nelsonhurstdev]

IMO, It can be removed. It is redundant now that the server has it integrated.

[Azeirah]

Slightly off-topic, but what does api_like_OAI.py currently offer that server does not?

I was thinking of removing that script since I believe server already support the OAI API

I don't think it offers anything extra anymore. Only thing I do know is that even today many people (I see it on reddit /r/LocalLLama and on LLM discords) don't know that the built-in server supports OAI endpoints.

Perhaps it shouldn't be removed but just give a big fat warning "PLEASE MOVE TO ./server THIS SCRIPT WILL BE REMOVED AT x DATE"

[AFKler]

Slightly off-topic, but what does api_like_OAI.py currently offer that server does not?

I was thinking of removing that script since I believe server already support the OAI API

server does not support the following arguments: --chat-prompt --user-name --ai-name --system-name
and README has not enough information on how to use --chat-template JINJA_TEMPLATE instead.