Optimise default_buffer_water_mark for memory allocation

[c-taylor]

Feature request.

The parameter proxy.config.http.default_buffer_water_mark can be used to increase memory buffering when reading out of the cache. This can have really positive properties with high levels of concurrency when using read while writer, but at the expense of memory. On a busy system with high concurrency, increasing this value even slightly can consume large amounts of memory, and it is impossible to assert how much of that is 'useful'.
'Useful' here is described as over buffering to both complete the write of the object, and to ensure the shared cursor is not dependent on the transfer rate of the first requester.

The positive effects are at their greatest when:

• Response size < default_buffer_water_mark
• Read while writer enabled
• High request concurrency

RAM cost for little to no benefit:

• Response size > buffer_water_mark

---

The suggestion here is to add a new parameter:
proxy.config.http.high_buffer_water_mark

We should defer increasing the buffer_water_mark above default , and ONLY if:
default_buffer_water_mark < "Content Length" < high_buffer_water_mark

This should have the desired improvement for all responses within the RAM buffering threshold we set, and not incur the cost for larger files outside of that window.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. Marking it stale to flag it for further consideration by the community.