Skip to content

Include a DECR benchmark covering embstring object #237

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 20, 2023
Merged

Conversation

filipecosta90
Copy link
Contributor

Reference:
redis/redis#12250

Simple run:

$ redis-benchmarks-spec-client-runner --test memtier_benchmark-1Mkeys-string-decr.yml  --override-memtier-test-time 10
2023-06-18 10:14:24 INFO redis-benchmarks-spec-client-runner (solely client) 0.1.71
2023-06-18 10:14:24 INFO Using test-suites folder dir /home/fco/.local/lib/python3.10/site-packages/redis_benchmarks_specification/test-suites
2023-06-18 10:14:24 INFO Running specific benchmark in 1 files: ['/home/fco/.local/lib/python3.10/site-packages/redis_benchmarks_specification/test-suites/memtier_benchmark-1Mkeys-string-decr.yml']
2023-06-18 10:14:24 INFO There are a total of 1 test-suites in folder /home/fco/.local/lib/python3.10/site-packages/redis_benchmarks_specification/test-suites
2023-06-18 10:14:24 INFO Overriding memtier benchmark --test-time to 10 seconds
2023-06-18 10:14:24 INFO Running the benchmark specs.
2023-06-18 10:14:24 INFO Loading default specifications from file: /home/fco/.local/lib/python3.10/site-packages/redis_benchmarks_specification/test-suites/defaults.yml
2023-06-18 10:14:24 INFO Found RedisTimeSeries default metrics specification. Will include the following metrics on all benchmarks $."BEST RUN RESULTS".Totals."Ops/sec" $."BEST RUN RESULTS".Totals."Latency" $."BEST RUN RESULTS".Totals."Misses/sec" $."BEST RUN RESULTS".Totals."Percentile Latencies"."p50.00" $."WORST RUN RESULTS".Totals."Ops/sec" $."WORST RUN RESULTS".Totals."Latency" $."WORST RUN RESULTS".Totals."Misses/sec" $."WORST RUN RESULTS".Totals."Percentile Latencies"."p50.00" $."AGGREGATED AVERAGE RESULTS (3 runs)".Totals."Ops/sec" $."AGGREGATED AVERAGE RESULTS (3 runs)".Totals."Latency" $."AGGREGATED AVERAGE RESULTS (3 runs)".Totals."Misses/sec" $."AGGREGATED AVERAGE RESULTS (5 runs)".Totals."Percentile Latencies"."p50.00" $."ALL STATS".Totals."Ops/sec" $."ALL STATS".Totals."Latency" $."ALL STATS".Totals."Misses/sec" $."ALL STATS".Totals."Percentile Latencies"."p50.00"
2023-06-18 10:14:24 INFO Found RedisTimeSeries default time metric specification. Will use the following JSON path to retrieve the test time $."ALL STATS".Runtime."Start time"
2023-06-18 10:14:24 INFO Benchmark required memory: 1073741824 Bytes
2023-06-18 10:14:24 INFO  Using total system memory as max 33346134016
2023-06-18 10:14:24 INFO Resetting commmandstats for shard 0
2023-06-18 10:14:24 INFO Test memtier_benchmark-1Mkeys-string-get-100B priority (98) is within the priority limit [0,100000]
2023-06-18 10:14:24 INFO Using docker image redislabs/memtier_benchmark:edge as benchmark PRELOAD image (cpuset=0,1,2,3) with the following args: /usr/local/bin/memtier_benchmark --port 6379 --server localhost --json-out-file oss-standalone-2023-06-18-09-14-24-NA-preload__memtier_benchmark-1Mkeys-string-get-100B.json  --command "SET __key__ 9223372036854775807" --command-key-pattern="P" -n 5000 --key-minimum=1 --key-maximum 1000000 -c 50 -t 4 --hide-histogram
2023-06-18 10:14:28 INFO Preload duration 4 secs.
2023-06-18 10:14:28 INFO Tool 4 seconds to load data. Output b'4         Threads\n50        Connections per thread\n5000      Requests per client\n\n\nALL STATS\n==================================================================================================\nType         Ops/sec    Avg. Latency     p50 Latency     p99 Latency   p99.9 Latency       KB/sec \n--------------------------------------------------------------------------------------------------\nSets       271621.77         0.72978         0.69500         1.45500         4.79900     17212.14 \nTotals     271621.77         0.72978         0.69500         1.45500         4.79900     17212.14 \n'
2023-06-18 10:14:28 INFO Benchmark used memory at start of benchmark: 1g
2023-06-18 10:14:28 INFO Checking if there is a keyspace check being enforced
2023-06-18 10:14:28 INFO Ensuring keyspace length requirement = 1000000 is met.
2023-06-18 10:14:28 INFO The total numbers of keys in setup matches the expected spec: 1000000==1000000
2023-06-18 10:14:28 INFO Will store benchmark json output to local file oss-standalone-2023-06-18-09-14-28-NA-memtier_benchmark-1Mkeys-string-get-100B.json
2023-06-18 10:14:28 INFO Using docker image redislabs/memtier_benchmark:edge as benchmark client image (cpuset=0,1,2,3) with the following args: /usr/local/bin/memtier_benchmark --port 6379 --server localhost --json-out-file oss-standalone-2023-06-18-09-14-28-NA-memtier_benchmark-1Mkeys-string-get-100B.json --test-time=10 --command "DECR __key__" --command-key-pattern="R" --key-minimum=1 --key-maximum 1000000 -c 4 -t 4 --hide-histogram
2023-06-18 10:14:39 INFO Benchmark duration 11 secs.
2023-06-18 10:14:39 WARNING Benchmark duration of 11 secs is bellow the considered minimum duration for a stable run (60 secs).
2023-06-18 10:14:39 INFO Printing client tool stdout output
2023-06-18 10:14:39 INFO Benchmark used memory at end of benchmark: 1g
2023-06-18 10:14:39 INFO Reading results json from /home/fco/tmpm5txmixj/oss-standalone-2023-06-18-09-14-28-NA-memtier_benchmark-1Mkeys-string-get-100B.json
# Results for memtier_benchmark-1Mkeys-string-get-100B test-case on oss-standalone topology
|                 Metric JSON Path                 |Metric Value|
|--------------------------------------------------|-----------:|
|"ALL STATS".Totals."Ops/sec"                      |  250742.070|
|"ALL STATS".Totals."Latency"                      |       0.064|
|"ALL STATS".Totals."Misses/sec"                   |       0.000|
|"ALL STATS".Totals."Percentile Latencies"."p50.00"|       0.063|
2023-06-18 10:14:39 INFO Using datapoint_time_ms: 1687079668434
2023-06-18 10:14:39 ERROR Requested to push data to RedisTimeSeries but no exporter definition was found. Missing "exporter" config.
2023-06-18 10:14:39 INFO Collecting memory metrics
2023-06-18 10:14:39 INFO Adding a total of 35 server side metrics collected at the end of benchmark
2023-06-18 10:14:39 INFO Collecting commandstat metrics
2023-06-18 10:14:39 INFO Adding a total of 20 server side metrics collected at the end of benchmark
2023-06-18 10:14:39 INFO Tearing down setup
2023-06-18 10:14:39 INFO Removing temporary client dir /home/fco/tmpm5txmixj
# Results for entire test-suite
|               Test Name                |                 Metric JSON Path                 |Metric Value|
|----------------------------------------|--------------------------------------------------|-----------:|
|memtier_benchmark-1Mkeys-string-get-100B|"ALL STATS".Totals."Ops/sec"                      |  250742.070|
|memtier_benchmark-1Mkeys-string-get-100B|"ALL STATS".Totals."Latency"                      |       0.064|
|memtier_benchmark-1Mkeys-string-get-100B|"ALL STATS".Totals."Misses/sec"                   |       0.000|
|memtier_benchmark-1Mkeys-string-get-100B|"ALL STATS".Totals."Percentile Latencies"."p50.00"|       0.063|

@codecov-commenter
Copy link

codecov-commenter commented Jun 18, 2023

Codecov Report

Patch and project coverage have no change.

Comparison is base (ead8bdd) 56.71% compared to head (712282c) 56.71%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #237   +/-   ##
=======================================
  Coverage   56.71%   56.71%           
=======================================
  Files          24       24           
  Lines        2234     2234           
=======================================
  Hits         1267     1267           
  Misses        967      967           

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@filipecosta90 filipecosta90 merged commit d9c42fc into main Jun 20, 2023
@filipecosta90 filipecosta90 deleted the decr.incr branch June 20, 2023 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants