v2.0: increase writeable accounts cost hashmap size (backport of #3009) #3019
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
During block packing, we maintain a hashmap that tracks the CUs for each writeable account. We constrain this to 12M CUs for a given account because we know accessing the same account restricts parallelism, and serializing too many operations may inflate slot times beyond 400ms.
We currently pre-allocate this hashmap with a capacity of 512. Not sure where exactly this number came from, but looks like it was chosen ~3 years ago.
Looking at mainnet over the last 7 days, we can see that we touch an average of 2.5-3k accounts and a max of ~5k (w/ spikes beyond 8k):
As soon as we go beyond the 512 pre-allocation, we have to resize the hashmap, which is costly.
Summary of Changes
Increase the default allocation to 4k so that we don't have to resize most of the time. Even at this size, the hashmap should be only 128kB or so. It might make sense to go even larger so we almost never have to resize (maybe 6k or 8k)
This is an automatic backport of pull request #3009 done by Mergify.