Stream to consumer while chunk is downloading

[abhijat]

When a chunk is downloading, we block the consumer until the download is finished. Once done, we read from disk and serve the fetch request. Instead of this, we should be able to serve the fetch request while downloading, from the same stream.

JIRA Link: CORE-1399

[abhijat]

This might need some rework based on recent discussions. The current approach in #12837 relies on stream while downloading. Another possible approach suggested by the team:

• stream the initial request
• once the current fetch completes (due to overbudget etc) download the remaining data to disk, so that when the consumer returns it is served disk based content

[toddfarmer]

Per discussion with @piyushredpanda , this is being set to Backlog. There's no associated Aha feature that might inform stakeholders - @mattschumpert , please review if any coordination is required.