gzip: Allocate gzip buffers from storage #4307
Open
+39
−20
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Arbitrary allocation from storage was introduced to solve the h2
head-of-line blocking caused by early DATA frames, allowed by the
initial control flow window of new streams. The decision could have
been made then to allocate the window buffer from the heap, but it
was tied to storage instead, offering better control over the global
memory footprint of the cache process.
The gzip buffer is tied to a task, but too large to allocate from
workspace without posing a risk. Even worse, there may be multiple
concurrent gzip operations in a single task. For example a gunzip
needed to parse ESI followed by a gzip to store a compressed body.
For a similar reason, it was not opportune to allocate the h2 stream
window buffer from workspace, despite being tied to the task.
Following the same logic, the gzip allocation can be performed from
storage to remove one wild card in our heap consumption. Another
possibility could have been the addition of a mempool, and it could
also have been an alternative for the h2 stream window, but transient
storage seemed more appropriate and it matches the on-demand malloc
behavior.
The gzip buffer logic could have been better encapsulated, but the
amount of direct access to the m_buf field would have resulted in a
lot of noise. Most of the noise here is caused by the two function
signatures changed to take a worker argument.