[💡FEATURE REQUEST]: Streaming request [WIP] & responses [HTTP-DONE]

[lwc]

This project looks like it could be very useful in building a GRPC server bridge for PHP, which would be amazing.

GRPC has the concept of streaming request & responses, is that something you'd consider adding to roadrunner/goridge?

Thanks

---

This ticket (since most of the users know it) split into the four parts:

• HTTP response streaming [done in v2023.3]
• HTTP request streaming
• gRPC response streaming
• gRPC request streaming

[wolfy-j]

It can be done by locking done and working with one worker within the request. This is definitely something we are going to consider adding going forward.

[wolfy-j]

I have put a foundation for streaming responses into a newer version. Streaming requests will be looked at once this feature it complete. See #9

[wolfy-j]

This will be coming in a separate server due async mode of underlying workers.

[pwilsoncentre]

I would love to see streaming support as well, but for large HTTP responses. I noticed this issue was reopened a few days ago, so does that mean that some progress will be made here? If I had time to learn Go I'd offer to help, but as it is, I can only offer my encouragement - this would be a really cool feature to have.

[wolfy-j]

Yes, this issue will be reconsidered for 1.5.0. No timeline yet but there are some initial experiments already made.

[wolfy-j]

Not in 1.5.0. We are working on a protocol update (with few extra features and perf improvements), so most likely it will be 2.0.0.

[Ocramius]

Ha, this is quite a roadblock: I was trying to migrate an existing system to RR this weekend, and one of the payloads that we stream is a GIGANTIC XML Sitemap that needs to be produced chunk-by-chunk (especially important to avoid running into memory limits).

I'm unsure about whether this is to be done in the GO components, or in the PHP-side of things (PSR-7 stream response missing there too, perhaps?).

I haven't done a detailed analysis, but this is indeed the biggest limitation I faced so far with the tool, whereas everything else is, I must say, quite impressive :-)

[wolfy-j]

With RR2 refactor we might be able to take a look at this issue with a fresh approach. We will discuss this internally to see if it is possible.

[rustatian]

Discussion: Local pipelining with jobs + broadcast plugin

[wolfy-j]

Essentially we have an idea of how to solve this issue with:

1. No memory leaks
2. No blocking on HTTP worker
3. Proper load balancing and prevention of over saturation
4. Side effect - extra feature reliable long polling in PHP
5. Horizontal scalability

[wolfy-j]

@rustatian we should consider doing the same for GRPC (streaming).