Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a launch flag for --latent-preview-api to get image previews through json payload #4993

Conversation

RandomGitUser321
Copy link
Contributor

@RandomGitUser321 RandomGitUser321 commented Sep 20, 2024

IGNORE THIS PR: I figured it out and made a separate PR updating the websocket example. See #5016

I've been trying to make a super simple Gradio frontend for the family to use, that's also phone/tablet friendly but found it really needed previews. I was driving myself insane trying to figure out how to decode the binary stream on the client app, so I just added an option to base64 encode the images and send them through json.

Here's an example of the previews working with a Gradio frontend that runs ComfyUI API workflows:
image

And then the final output:
image

I should also note that it seems to trigger websocket timeouts a little more frequently than normal, but that would make sense with it hammering images through json. In my case, I just added a timeout to my app and if it does so, I can just click generate and it works again. Taesd previews also work, but since they are usually full size, it would mean more data to send.
I think I solved the timeout errors I'd sometimes get and it turns out it was a kind of stupid mistake:
I wasn't using ws.close() at the end of my websocket call function to ComfyUI... Been slamming it with a bunch of generations and taesd previews, haven't had a websocket timeout since!

All that being said, this is an opt-in type option that should have no effect on the rest of ComfyUI if you aren't specifically trying to use it. Ideally, I should also eventually add in a server option to throttle the rate it sends these images as well, but I'll have to look into how/where to put it.

Realized that it still needs the regular unecoded_preview_image as well, in order to get previews on another device
@RandomGitUser321
Copy link
Contributor Author

RandomGitUser321 commented Sep 21, 2024

Slight update on it: I realized that it still needs the regular unecoded_preview_image as well, if you also plan on wanting to have node previews within comfy from another device. So now, if the launch flag is there, it will send both the unecoded_preview_image and the base64 version and you can have previews in both comfy and an app using the api.

@asagi4
Copy link
Contributor

asagi4 commented Sep 21, 2024

@RandomGitUser321 I'm interested in what the problem is with decoding the raw data? Does the websocket send something other than the raw bytes of the image file over the network?

@RandomGitUser321
Copy link
Contributor Author

RandomGitUser321 commented Sep 21, 2024

SOLVED it and put it in a new PR: #5016

@RandomGitUser321 RandomGitUser321 deleted the latent_preview_api branch September 22, 2024 04:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants