Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use streaming and other techniques to improve latency and responsiveness #51

Open
MrCsabaToth opened this issue Sep 14, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@MrCsabaToth
Copy link
Member

Felipe Gallo's advice for Omni applies to Open MMPA too:
https://discuss.ai.google.dev/t/seeking-feedback-on-omni/38695/3?u=tocsa

  • Instead of waiting the speech and or text, stream the response and take action with partial information (if possible)
  • With the same idea, if possible, stream the speech to text also. Don’t wait until the user stops talking to process all the information
  • Distract the user with something while your app is doing background tasks (some loading animation or even some status like “Opening firefox… opening instagram…”
  • Answer something instantly after the user command like “Alright, im already working on that”. This will make tasks that took 3 seconds looks like took 2 (because the awkwardness is in the silence without doing nothing/saying nothing)
  • Consider using parallel processing and race conditions for tasks that are not intrinsicaly related for example: “search online for dinosaur images (parallel process 1) create a folder with a specific name on my desktop (parallel process 2), and then build a website using those images along with some related information using HTML and JavaScript (wait for 1 and 2 and then do this task)”
@MrCsabaToth MrCsabaToth added the enhancement New feature or request label Sep 14, 2024
@MrCsabaToth MrCsabaToth changed the title Use streaming and other techniques to improve latency and be more responsive Use streaming and other techniques to improve latency and responsiveness Sep 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant