[BUG]: Fail after memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values.yml

[markovamaria]

Describe the bug
Tests behavior is broken after run of test "memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values.yml":
1st run returns results, 2nd failed, 3rd run kills server (and returns strange 0 results for Ops/sec).

To Reproduce
Run server:

$ ./src/redis-server --port 6379 --dir ~/redis_server_logs --logfile fail.log  --dbfilename fail.rdb

In parallel run test "memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values.yml" twice, second run is failing + empty tmp* folder is appeared.

$ redis-benchmarks-spec-client-runner --db_server_host localhost --db_server_port 6379 --client_aggregated_results_folder ./test1 --test memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values.yml
...
# Results for memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values test-case on oss-standalone topology
|                 Metric JSON Path                 |Metric Value|
|--------------------------------------------------|-----------:|
|"ALL STATS".Totals."Ops/sec"                      |   92584.090|
|"ALL STATS".Totals."Latency"                      |       2.154|
|"ALL STATS".Totals."Misses/sec"                   |       0.000|
|"ALL STATS".Totals."Percentile Latencies"."p50.00"|       2.127|
# Results for entire test-suite
|                          Test Name                           |                 Metric JSON Path                 |Metric Value|
|--------------------------------------------------------------|--------------------------------------------------|-----------:|
|memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values|"ALL STATS".Totals."Ops/sec"                      |   92584.090|
|memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values|"ALL STATS".Totals."Latency"                      |       2.154|
|memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values|"ALL STATS".Totals."Misses/sec"                   |       0.000|
|memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values|"ALL STATS".Totals."Percentile Latencies"."p50.00"|       2.127
...

$ redis-benchmarks-spec-client-runner --db_server_host localhost --db_server_port 6379 --client_aggregated_results_folder ./test2 --test memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values.yml
...
2022-05-13 21:47:56 CRITICAL Some unexpected exception was caught during local work. Failing test....
2022-05-13 21:47:56 CRITICAL <class 'redis.exceptions.ResponseError'>
2022-05-13 21:47:56 INFO Tearing down setup
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ubuntu/Work/env3/lib/python3.8/site-packages/redis_benchmarks_specification/__runner__/runner.py", line 260, in process_self_contained_coordinator_stream
    r.ping()
  File "/home/ubuntu/Work/env3/lib/python3.8/site-packages/redis/commands/core.py", line 1134, in ping
    return self.execute_command("PING", **kwargs)
  File "/home/ubuntu/Work/env3/lib/python3.8/site-packages/redis/client.py", line 1227, in execute_command
    return conn.retry.call_with_retry(
  File "/home/ubuntu/Work/env3/lib/python3.8/site-packages/redis/retry.py", line 46, in call_with_retry
    return do()
  File "/home/ubuntu/Work/env3/lib/python3.8/site-packages/redis/client.py", line 1228, in <lambda>
    lambda: self._send_command_parse_response(
  File "/home/ubuntu/Work/env3/lib/python3.8/site-packages/redis/client.py", line 1204, in _send_command_parse_response
    return self.parse_response(conn, command_name, **options)
  File "/home/ubuntu/Work/env3/lib/python3.8/site-packages/redis/client.py", line 1243, in parse_response
    response = connection.read_response()
  File "/home/ubuntu/Work/env3/lib/python3.8/site-packages/redis/connection.py", line 842, in read_response
    raise response
redis.exceptions.ResponseError: MISCONF Redis is configured to save RDB snapshots, but it's currently unable to persist to disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error.
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ubuntu/Work/env3/bin/redis-benchmarks-spec-client-runner", line 8, in <module>
    sys.exit(main())
  File "/home/ubuntu/Work/env3/lib/python3.8/site-packages/redis_benchmarks_specification/__runner__/runner.py", line 129, in main
    process_self_contained_coordinator_stream(
  File "/home/ubuntu/Work/env3/lib/python3.8/site-packages/redis_benchmarks_specification/__runner__/runner.py", line 509, in process_self_contained_coordinator_stream
    local_benchmark_output_filename,
UnboundLocalError: local variable 'local_benchmark_output_filename' referenced before assignment

3rd run returns broken? results, but Kills server.

$ export exp=fail3 && redis-benchmarks-spec-client-runner --db_server_host localhost --db_server_port 6379 --client_aggregated_results_folder ./"$exp" --test memtier_benchmark-10Mkeys-load-ha
sh-5-fields-with-1000B-values.yml |& tee fail3.log
...
2022-05-13 21:51:20 INFO Storing an aggregated results CSV into /home/ubuntu/tmp90c87hic/oss-standalone-2022-05-13-21-49-53-NA-memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values.json

# Results for memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values test-case on oss-standalone topology
|                 Metric JSON Path                 |Metric Value|
|--------------------------------------------------|-----------:|
|"ALL STATS".Totals."Ops/sec"                      |       0.000|
|"ALL STATS".Totals."Latency"                      |       2.266|
|"ALL STATS".Totals."Misses/sec"                   |       0.000|
|"ALL STATS".Totals."Percentile Latencies"."p50.00"|       2.175|
# Results for entire test-suite
|                          Test Name                           |                 Metric JSON Path                 |Metric Value|
|--------------------------------------------------------------|--------------------------------------------------|-----------:|
|memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values|"ALL STATS".Totals."Ops/sec"                      |       0.000|
|memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values|"ALL STATS".Totals."Latency"                      |       2.266|
|memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values|"ALL STATS".Totals."Misses/sec"                   |       0.000|
|memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values|"ALL STATS".Totals."Percentile Latencies"."p50.00"|       2.175| 

AND

$ ./src/redis-server --port 6379 --dir ~/redis_server_logs --logfile fail.log  --dbfilename fail.rdb
Killed

Environment (please complete the following information):

• OS: ubuntu 20.04
• PyPi version package - latest
• Host: AWS instance - m6i.4xlarge

[markovamaria]

Is not reproducible on latest versions:

bench's version 57 (latest available in pip)
redis-server version:
commit f36eb5a1ba81f7f5522a5e43000d3e6997651e12 (HEAD -> unstable, origin/unstable, origin/HEAD)
Author: Itamar Haber itamar@redis.com
Date: Tue Nov 22 18:10:21 2022 +0200"

[markovamaria]

Was able to make test fail after series of runs with error:

2022-11-23 16:43:57 INFO Reading results json from /home/berta/tmpamnzxlrl/oss-standalone-2022-11-23-15-38-40-NA-memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values.json
2022-11-23 16:43:57 CRITICAL Some unexpected exception was caught during local work. Failing test....
2022-11-23 16:43:57 CRITICAL <class 'json.decoder.JSONDecodeError'>
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/berta/mmarkova/env_redis/lib/python3.7/site-packages/redis_benchmarks_specification/__runner__/runner.py", line 504, in process_self_contained_coordinator_stream
    results_dict = json.load(json_file)
  File "/home/berta/data/miniconda3/envs/py3.7.6/lib/python3.7/json/__init__.py", line 296, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/home/berta/data/miniconda3/envs/py3.7.6/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/home/berta/data/miniconda3/envs/py3.7.6/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/berta/data/miniconda3/envs/py3.7.6/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1677 column 26 (char 35942)
------------------------------------------------------------
2022-11-23 16:43:57 INFO Tearing down setup
2022-11-23 16:43:57 INFO Preserving local results file /home/berta/tmpamnzxlrl/oss-standalone-2022-11-23-15-38-40-NA-memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values.json into ./test1/oss-standalone-2022-11-23-15-38-40-NA-memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values.json
2022-11-23 16:43:57 INFO Removing temporary client dir /home/berta/tmpamnzxlrl
# Results for entire test-suite
|Test Name|Metric JSON Path|Metric Value|
|---------|----------------|------------|
2022-11-23 16:44:02 INFO Storing an aggregated results CSV into /home/berta/tmpamnzxlrl/oss-standalone-2022-11-23-15-38-40-NA-memtier_benchmark-10Mkeys-load-hash-5-fields-with-1000B-values.json

Run server with:

 ./src/redis-server --port 6379 --dir run_server_logs --logfile run_server.log --dbfilename fail1.rdb

[markovamaria]

Wrong execution way (--dbfilename was used) lead to errors.
Should be run with --save no.
Update to readme is suggested.

Closing issue as not relevant.