transport/bufWriter: fast-fail on error returned from flushKeepBuffer()

[veshij]

fixes #7389

RELEASE NOTES:

• transport: Fix a bug which could result in writes busy looping when the underlying conn.Write returns errors

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.42%. Comparing base (bdd707e) to head (877dd6f).
Report is 38 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff            @@
##           master    #7394    +/-   ##
========================================
  Coverage   81.42%   81.42%            
========================================
  Files         348      354     +6     
  Lines       26744    27083   +339     
========================================
+ Hits        21775    22053   +278     
- Misses       3779     3817    +38     
- Partials     1190     1213    +23     

Files	Coverage Δ
internal/transport/http_util.go	93.30% <100.00%> (+6.89%)	⬆️

... and 41 files with indirect coverage changes

[arjan-bal]

Thank you for sending the fix, left some minor comments.

[arjan-bal]

LGTM, adding another reviewer for a second approval.

[easwars]

We probably can add a test table, where we create the bufWriter with and without a shared pool and ensure that the behavior works across both cases.

[easwars]

Also, please respond to individual comments. That helps reviewers to keep track of the individual changes that they requested (and eventually mark them as resolved). Thanks.

[easwars]

Looks like a bunch of tests are failing. PTAL.

[veshij]

looking at failed tests

[dfawley]

@veshij any update?

[veshij]

@veshij any update?

@dfawley
*bufWriter.Write implementation can completely skip flushing if w.offset >= w.batchSize condition is never met.
In this case once this buffer is reused later on - flushKeepBuffer() would return more bytes than was initially sent to Write() call. Thus we can't reuse value returned from flushKeepBuffer().
One option would be to leave total written bytes accounting as is (always written += copied), or force-flush buffer at the end of Write call (which is likely too expensive).

Updated PR with the first option.

[easwars]

@printchard : FYI