Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change the default values for generation in GUI #840

Merged
merged 1 commit into from
Jun 9, 2023

Conversation

pingpongching
Copy link
Contributor

@pingpongching pingpongching commented Jun 5, 2023

the defaults that we have right now aren't ideal, i tried many models(nous-vicuna, snoozy, groovy....) with the values given here: https://www.reddit.com/r/LocalLLaMA/wiki/index/#wiki_prompting , i used the "Precise" preset, and all of the models gave much better results.
Amount of times the models got stuck in a loop writing the same thing over and over significantly decreased using the new values, that allowed to make them write more longer responses, and follow the conversation for longer, without going schitzo.
And i believe the newcomers will appreciate not having to figure out why the models are constantly getting stuck in loops whenever the conversation gets a little long.

Also changed the batch size from 9 to 128, 9 is too conservative imo.

@AndriyMulyar
Copy link
Contributor

AndriyMulyar commented Jun 5, 2023

We would need some heavy testing to make these the defaults. Ideally, we should be sending per model defaults by specifying them in the models.json file.

@manyoso
Copy link
Collaborator

manyoso commented Jun 5, 2023

The current settings were mostly on the advice/direction of some people in discord and some scant testing I did early on. We currently don't have a good way to objectively test default values across our model zoo. I'm all for letting people experimenting and letting the hive mind decide what our defaults should be at this point. @Andriy can you do some kind of poll in discord maybe?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants