Extensions for volume testing #223
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Try add a driver thta does a balance of NHS style work
# Conflicts: # examples/riakc_pb.config
format is not as suggested in erlang client code comments
To give a balance between uniform int and pareto int
Fixed y axis, as well as simplified layout
including logging of stop and start
Cannot use listkeys stats, and so will log timer for each listkeys separately
Idea is to run a separate basho_bench process for testing list keys - that will pause between requests
For listkeys
Only one of the workers will run listkeys
Difficult to be sure that behaviour seen in test is as a result of feedback loop if listkeys takes longer (i.e. less requests are sent), rather than the actual cluster reacting to the ongoing listkeys operation. So now try and make this listkeys request async (but still ensure only one is run at a time)
When checking is_process_alive
Allows for the frequency of particularly large objects to be enhanced
Reduce the relative frequency of regex queries
get_unique was never getting an object ... always a miss ... no wonder performance improved!
To something more realistic
This seems more realistic
Allows for a GET every 10 seconds which will prompt an AAE exchange
Stop having get requests where not_found is an expected response. This will slow throughput in early parts of the test - but should make sure the tests are more relaistic, not being dominated by the speed of not_founds for multiple hours.
In testing - use a 24 hour config with more workers
Otherwise they will throw errors when processing 2i query results
AboveMin was BelowMin, and due to 80/20 rule was not frequently being triggered - too many additions, not enough updates. Also too high a proportion of updates compared to GETs. Need to avoid running out of disk space in test environment, so overall key space size reduced.
Disk space ran out 18 hours into the test. Should be enough to reduce proportion of updates, and reduce key space by 20% (not 40%)
This reverts commit c95516c.
Not every increment below 1000. D'oh.
clincials environment has a different profile - add a specific basho_bench for this
Due to disk space issues
So that there is a natural pre-load stage up to a minumum database size
And allow for change of value size. clinicals documents ar enot base don internalID, and so will be in a random order
To distnguish between different nodes use a key id rather than random bytes. Aim is to make different nodes produce different keys - if we run basho_bench from > 1 test harness
General load test. Sort of somewhere between the demands of all three functional domain clusters. Also included is a passive version, designed to be a relatively unchallenging passive-site load (for providing load under cluster admin tests).
|
This is what has been used for 2.9 volume testing |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New bench drivers to allow for:
Better volume testing of 2i;
Test runs that first load a majority of PUTs to build up a baseline of data, before switching to majority GETs;
Variations on pareto distributions;
Alternative value generators to make compressed, semi-compressible objects