Add QueryAgent streaming

[danmichaeljones]

Adds an option to stream from the QueryAgent, by consuming the server-sent-events API on the backend. This new method yields objects providing progress messages (e.g., Analyzing Query...) as well as per-token streaming of the final answer, before the final state (like from .run()).

This new method can be used as follows:

const qa = new QueryAgent(...);
const query = "..."

for await (const event of qa.stream(query)) {
    if (event.output_type === "progress_message") {
        // The message is a human-readable string, structured info available in event.details
        console.log(event.message);
    } else if (event.output_type === "streamed_tokens") {
        // The delta is a string containing the next chunk of the final answer
        process.stdout.write(event.delta);
    } else {
        // This is the final response, as returned by QueryAgent.run()
        event.display();
    }
}

[orca-security-eu]

Orca Security Scan Summary

Status	Check	Issues by priority
Passed	Secrets	0   0   0   0	View in Orca

[augustas1]

have you checked https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events?
looks very easy to implement

[danmichaeljones]

Yeah, I checked those docs first, and started implementing this using EventSource. The problem with it is that EventSource can only send GET requests (see e.g., here, here), whereas our API expects a POST request with a JSON body.

[augustas1]

ah, that's a shame :/