FR: Streaming request

[svilupp]

It would be excellent to add streaming as an option for OpenAI requests.

Given how long it takes to generate a response, seeing the partial response coming through would be better than a progress bar (and allows users to react/think through the response).

Server-Sent Events are implemented in OpenAI API and their cookbook has examples for the Python SDK SSE

We should be able to achieve the same with HTTP.jl, see Request Streaming.

As always, I’m happy to take a stab at it if there is interest.

[ThatcherC]

I have some code for this I was developing off of v0.6.0! I'll try to bring it up to v0.8.0 and put out a PR. I didn't quite figure out how to make the callbacks work nicely but it might not be too far off.

[svilupp]

Awesome! I look forward to testing it out - it would make the wait so much less painful!

My next stop is to build out the self-iterative loops (ala CodeT/Reflexion) and, potentially, parallelize different tasks to speed up time to a ready solution.

[roryl23]

This is a fantastic idea