Add BANIDX facility to speed up BAN loading #4261
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When loading a cache with a relevant number of BANs from a persistent storage,
BAN_FindBan()
needs to called for every object to determine the struct ban which the in-core object is to point to based on the persisted ban time.This is highly inefficient because
BAN_FindBAN()
conducts a linear search of the ban list, for which we would expect about half the bans to need inspection if objects were distributed equally across all bans. But for a long BAN list, most objects will hang near the bottom, so chances are high that we actually need to traverse nearly all of the ban list for each loaded object.So this screams for turning the ban list into a different data structure, like a binary tree.
Yet once loading of objects is complete, the ban list as we have it is optimal: All operations on the ban list need to traverse it anyway and in many cases this traversal can happen unlocked. So we really would not want to change the structure of the ban list.
This patch suggests to add a search tree as an additional data structure, which only exists during cache load (while there are
ban_hold
s). In particular, the interface is made such that the only relevant change to the ban code is to provide a hint for the linear search to start at:VTAILQ_FOREACH(ban, ...)
gets turned intoand that's it.
In particular, this patch implies no change at all to setups which do not use persistent storage, so the risk is considered very low.
The speed up is impressive:
The following tests were conducted with a SLASH/fellow storage with 7689 bans. For each test round, the storage (residing on a ZFS volume) was rolled back to the same state, basically like so:
When the cache was loaded, varnishd was terminated and the test repeated.
Six runs were conducted with Varnish-Cache master as of 749a2c3, resulting in load times in the order of 21 seconds:
(note: the change of the number of cache objects is because wall clock time has passed the expiry time of one object)
With this patch, load times were reduced by a factor >15
Naturally, the benefits are even more relevant with a higher number of bans.