HTTP2 FlowControl Window depleted

[AlfredoG87]

Given several minutes streaming data to a client, Helidon 4.x Http2 Outbound Flow Control always ends up with a negative remainingWindowSize and the thread will remain trapped in an infinite loop waiting for WINDOW_UPDATEs from the client

This happens on an outbound grpc stream, for easy replication we modified one of your examples for a gRPC WebServer with the following changes.

1. The Split stream will stream the first word indefinitively.
2. We commented out the test for such method, since we changed the behaviour of the method to showcase the bug.

You can see the full source code here --> https://github.com/AlfredoG87/helidon-examples/tree/helidon-4.x/examples/webserver/grpc

Environment Details

• Helidon Version: 4.1.4
• Helidon SE
• JDK version: 21
• OS: MacOs

---

Problem Description

The WebServer (gRPC) stops streaming after a short amount of time.

Does not appears to happen always in the same way, sometimes it takes longer to happen, but does always happens, only precondition is for outbound stream messages to be large and sustained.

You can see a video showing how the window size becomes negative, never recovers and the server stops streaming on the attached video.

Helidon.Http2.FlowControl.Window.Depleted.mp4

Steps to reproduce

Fork the following project: https://github.com/AlfredoG87/helidon-examples/tree/helidon-4.x/examples/webserver/grpc

Or apply the following commit or changes to your own copy of helidon-examples repo.

Run application, consume Split method with at least 2 words as input, the first one a large amount of lenght.

[AlfredoG87]

I believe that this could be related to this bug as well: #9122

[danielkec]

Hi, @AlfredoG87, can you please share what client you have used for the step:
consume Split method with at least 2 words as input, the first one a large amount of lenght. ?

[AlfredoG87]

Hi, @AlfredoG87, can you please share what client you have used for the step: consume Split method with at least 2 words as input, the first one a large amount of lenght. ?

In the video shared you can see I used Postman however we have also encountered this issue with grpcurl and netty shaded

[danielkec]

Sorry I can't download the video

[AlfredoG87]

Sorry I can't download the video

np, let me know if you know any more info. should be really straight forward to replicate. do you want me to convert the video to a gif or something else?

[danielkec]

I can reproduce it now with postman, thx for the video. Good news is that it is already fixed by #9520 (splitting large frames) issue. Bad news is it is not released yet.

[AlfredoG87]

Sorry I can't download the video

np, let me know if you know any more info. should be really straight forward to replicate. do you want me to convert the video to a gif or something else?

ohh no worries, just knowing that is already fixed is great news!! 💯

just to plan accordingly, do you have an approximate ETA of when we can see that fix released?

Lastly, want me to close the issue?

[danielkec]

Keep it open, I have added it to 4.2.0 milestone, and we will consult internally if we can release 4.1.x bugfix earlier

[danielkec]

Any chance you could try out the fix from a snapshot build?

[AlfredoG87]

Any chance you could try out the fix from a snapshot build?

Yes, I can, can you provide the link to the snapshot build? please

Update: I think I found it is just 4.1.5-SNAPSHOT version right?

[danielkec]

I don't think we have such snapshot build(4.2.0-SNAPSHOT), but it should be easy to build with JDK21+

cd /tmp && git clone git@github.com:helidon-io/helidon.git && cd helidon && mvn install -DskipTests

[AlfredoG87]

I don't think we have such snapshot build(4.2.0-SNAPSHOT), but it should be easy to build with JDK21+

Perfect, I can manage that!

[mattp-swirldslabs]

Hi @danielkec, I've been working with @AlfredoG87 to troubleshoot this issue on our side. I was able to confirm the inbound fixes in 4.2.0-SNAPSHOT with our server implementation. Thanks for pointing us to that PR and for all the changes. This helps unblock our project. 🙂 We're eagerly awaiting 4.2.0!

[AlfredoG87]

@danielkec Thank you for your help!! 💯

[AlfredoG87]

@m0mus just one question, has this already been release? 4.1.5 or when is it scheduled to be released in 4.1.6?

[danielkec]

Hi, sorry for not notifying you earlier. It will be part of 4.1.6 possibly tomorrow

[AlfredoG87]

No problem, I just asked since saw it was closed! 👍
TYVM