Skip to content

Conversation

@Gipphe
Copy link
Collaborator

@Gipphe Gipphe commented Jan 20, 2026

Track bytes in flight for current block fetch requests on a per-node basis and limit the bytes in flight accordingly. This strategy is adopted from the cardano-node's own implementation of watermarks and in-flight byte tracking for BlockFetch.

Depends on #200

@Gipphe Gipphe force-pushed the refactor/vertically-organize-peersharing branch from 089f855 to d77b2a4 Compare January 21, 2026 14:09
@Gipphe Gipphe force-pushed the feat/track-block-fetch-bytes-in-flight branch from b318cd4 to e5dbbd2 Compare January 21, 2026 14:35
@Gipphe Gipphe force-pushed the refactor/vertically-organize-peersharing branch from d77b2a4 to 0762f22 Compare January 21, 2026 16:15
@Gipphe Gipphe force-pushed the feat/track-block-fetch-bytes-in-flight branch from e5dbbd2 to 6c878c7 Compare January 22, 2026 07:18
@Gipphe Gipphe force-pushed the refactor/vertically-organize-peersharing branch 2 times, most recently from 507e5e0 to b38a4c8 Compare January 22, 2026 10:03
@Gipphe Gipphe force-pushed the feat/track-block-fetch-bytes-in-flight branch from 6c878c7 to 105626a Compare January 22, 2026 10:03
@Gipphe Gipphe force-pushed the refactor/vertically-organize-peersharing branch from b38a4c8 to 92e93e6 Compare January 22, 2026 10:05
@Gipphe Gipphe force-pushed the feat/track-block-fetch-bytes-in-flight branch from 105626a to 4e82322 Compare January 22, 2026 10:05
@Gipphe Gipphe force-pushed the refactor/vertically-organize-peersharing branch from 92e93e6 to 36a39d0 Compare January 22, 2026 10:52
@Gipphe Gipphe force-pushed the feat/track-block-fetch-bytes-in-flight branch 2 times, most recently from 0e75bd2 to aad0d3f Compare January 22, 2026 11:58
@Gipphe Gipphe force-pushed the refactor/vertically-organize-peersharing branch from 36a39d0 to 170066b Compare January 22, 2026 11:58
@Gipphe Gipphe force-pushed the feat/track-block-fetch-bytes-in-flight branch from aad0d3f to 22d4eca Compare January 23, 2026 08:30
@Gipphe Gipphe force-pushed the refactor/vertically-organize-peersharing branch from 170066b to 152eefd Compare January 23, 2026 08:30
@Gipphe Gipphe force-pushed the feat/track-block-fetch-bytes-in-flight branch from 22d4eca to 77ee248 Compare January 23, 2026 11:18
@Gipphe Gipphe force-pushed the refactor/vertically-organize-peersharing branch from 152eefd to 347de2d Compare January 23, 2026 11:18
@Gipphe Gipphe force-pushed the feat/track-block-fetch-bytes-in-flight branch from 77ee248 to aa7e3a5 Compare January 23, 2026 13:56
@Gipphe Gipphe force-pushed the refactor/vertically-organize-peersharing branch from 347de2d to 877af59 Compare January 23, 2026 13:56
@Gipphe Gipphe force-pushed the feat/track-block-fetch-bytes-in-flight branch from aa7e3a5 to 769eeea Compare January 23, 2026 15:03
@Gipphe Gipphe force-pushed the refactor/vertically-organize-peersharing branch 2 times, most recently from 780e2bf to 5bcdf6b Compare January 26, 2026 07:50
@Gipphe Gipphe force-pushed the feat/track-block-fetch-bytes-in-flight branch from 769eeea to f2cf190 Compare January 26, 2026 07:51
@Gipphe Gipphe force-pushed the refactor/vertically-organize-peersharing branch from 5bcdf6b to ee4ce3f Compare January 26, 2026 07:52
@Gipphe Gipphe force-pushed the feat/track-block-fetch-bytes-in-flight branch 2 times, most recently from 9b69e71 to b65be97 Compare January 26, 2026 09:45
@Gipphe Gipphe force-pushed the refactor/vertically-organize-peersharing branch from ee4ce3f to d404350 Compare January 26, 2026 09:45
Base automatically changed from refactor/vertically-organize-peersharing to main January 26, 2026 09:49
@dpulls
Copy link

dpulls bot commented Jan 26, 2026

🎉 All dependencies have been resolved !

@Gipphe Gipphe force-pushed the feat/track-block-fetch-bytes-in-flight branch from b65be97 to 0759f6a Compare January 26, 2026 12:58
@Gipphe Gipphe force-pushed the feat/track-block-fetch-bytes-in-flight branch 6 times, most recently from 288ae0d to c81ddaa Compare January 27, 2026 11:33
@Gipphe Gipphe marked this pull request as ready for review January 27, 2026 13:31
@Gipphe Gipphe requested review from cgeorgii and prednaz January 27, 2026 13:32
Log.info "Waiting until ready to issue block fetch request..."
_ <- readMVar =<< gets (.stateVar)
Log.info "Ready to issue block fetch request!"
pure ()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pure ()

awaitMessage outChan = do
waitQSem handles.qSem

waitUntilReadyToFetch
Copy link
Member

@prednaz prednaz Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

waiting like this would only make sense if this mini protocol was executed concurrently in multiple threads. else, who are we waiting for? this mini protocol is not executed concurrently in multiple threads, is it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a valid point. The current block fetch client is not pipelined (unlike ChainSync), so I think this flow control mechanism is never going to trigger.

@Gipphe either we pipeline the client or reevaluate whether this PR benefits us at all. I'm leaning toward the latter for now.

@Gipphe Gipphe force-pushed the feat/track-block-fetch-bytes-in-flight branch from c81ddaa to bd16a22 Compare January 28, 2026 11:05
@cgeorgii
Copy link
Member

@Gipphe while reviewing I changed a few things to improve readability and opened a pull request targeting this branch. Please check and merge into this, then I'll approve it.

@Gipphe Gipphe force-pushed the feat/track-block-fetch-bytes-in-flight branch from bd16a22 to fcb22a8 Compare January 28, 2026 12:59
@Gipphe Gipphe force-pushed the feat/track-block-fetch-bytes-in-flight branch from fcb22a8 to 272b66a Compare January 28, 2026 13:01
@Gipphe Gipphe closed this Jan 28, 2026
@Gipphe Gipphe deleted the feat/track-block-fetch-bytes-in-flight branch January 28, 2026 13:57
@Gipphe
Copy link
Collaborator Author

Gipphe commented Jan 28, 2026

There is no pipelined version of the BlockFetch client, so I'm not entirely sure how cardano-node parallelizes their block fetching. Since just upping the maxIngressQueueLimit helped this situation, and we don't end up blowing the new limit with concurrent block fetch requests, we'll shelve this PR for now as archived/feat/track-block-fetch-bytes-in-flight.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants