Description
LocalAI version:
version":"v2.26.0 (09941c0)
Describe the bug
When querying for example "gpt-4" model at the endpoint http://localhost:8080/v1/chat/completions with {"stream": true} passed in the request body, the returned JSON object in the key-value pair: "finish_reason": "" returns an empty string. The official OpenAI API returns "finish_reason": null with the same query on same model.
When the "finish-reason" value is returned as an empty string in stream, using the OpenAI libraries in application throws an exception:
ArgumentOutOfRangeException
Message = "Unknown ChatFinishReason value. (Parameter 'value') Actual value was ."
In the OpenAI libraries, ChatFinishReason is a nullable enum type, that does not accept an empty string. Probably, the LocalAI API should also return the key-value: "finish_reason": null in such a situation.
To Reproduce
Prompt "gpt-4" model with key-value pair {"stream": true} in request body
Expected behavior
In JSON response return key-value "finish_reason": null