[Feature Request] Custom Sampling Settings

[jackyc2718]

Clear and concise description of the problem

Autoregressive model can easily stuck into a loop and output repetitive contents due to its design. While some sampling settings such as temperature and top-p can help to alleviate the problem, other sampling methods/settings may be more effectively.

I believe that the generation code of the demo uses only the default OpenAI generation settings, i.e. temperature = 1 and top_p = 1 as they are not specified. There is current no support for params other than the default values, nor the use of presence_penalty and frequency_penalty which are both native to the API.

I think adding this feature would allow us to experiment with different settings, and choose the best one depending on user case to improve experience.

Suggested solution

1. Add sliders in the UI for temperature, top-p, presence_penalty, frequency_penalty
2. Specify the above parameters in the generation code, API Reference - OpenAI API

Alternative

No response

Additional context

No response

Validations

• Follow our Code of Conduct
• Read the Contributing Guide.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.

[ddiu8081]

Added in v2.