Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance improvements #429

Merged
merged 12 commits into from
Nov 13, 2024
Merged

Conversation

art-from-the-machine
Copy link
Owner

  • For both Piper and XTTS, voice model loading functions are now called as soon as the first NPC is ready, and are run asynchronously while the LLM generates a response
  • Added option to Mantella UI to enable / disable lip file generation (or select "Lazy" to skip lip generation for the first returned sentence to improve response times)
  • Added tiktoken encoding model locally to data/ folder to avoid connecting to the internet
  • Async OpenAI client is created on init to improve response times at the start of a conversation
  • Minor performance improvements to conversation summaries loading, NPC searching in character_df, and "trust" calculation
  • Added checks to see if context values exist in JSON input before trying to load them
  • Added timing logs to many functions for better performance monitoring

@art-from-the-machine art-from-the-machine merged commit 3b307d2 into main Nov 13, 2024
@art-from-the-machine art-from-the-machine deleted the performance_improvements branch November 13, 2024 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant