Skip to content
Open
Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
9fc6dd0
fix(tests): Use architecture-specific streams in coordinator tests
slice4e Oct 16, 2025
c7c300f
style: Format test file with black
slice4e Oct 16, 2025
2003061
style: Format runner.py with black
slice4e Oct 16, 2025
baa28ea
Merge main to get runner.py black formatting fix
slice4e Oct 16, 2025
4592b97
fix(runner): Skip validation for untested operation types
slice4e Oct 16, 2025
d00eb32
fix(tests): Add git_version to benchmark stream requests
slice4e Oct 16, 2025
fac13f6
fix(cli): Extract git_version from Docker image tag
slice4e Oct 16, 2025
805826e
style: Format cli.py with black
slice4e Oct 16, 2025
773554d
debug: Add logging to diagnose artifact_version issue
slice4e Oct 16, 2025
1e76e7b
debug: Add print statements to track artifact_version value
slice4e Oct 16, 2025
7e88020
style: Format runner.py with black
slice4e Oct 16, 2025
0fdc3a4
fix: Include running_platform in by.version key paths
slice4e Oct 16, 2025
d341d1b
fix: Add JSON module support and correct test metadata
slice4e Oct 16, 2025
a1f82e5
Merge pull request #3 from slice4e/fix_test_self_contained_coordinato…
slice4e Oct 16, 2025
5bbbe06
fix missing quote from test case memtier_benchmark-1Mkeys-string-set-…
slice4e Oct 17, 2025
683cda5
Merge pull request #4 from slice4e/fix_tests
slice4e Oct 20, 2025
df82e52
fix: Add null check before shutil.copy to prevent NoneType error
slice4e Oct 20, 2025
0748c88
Fix multi-client tests not writing aggregated results to files
slice4e Oct 20, 2025
a56d26e
Merge pull request #5 from slice4e/fix-redis-bench-test
slice4e Oct 20, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix multi-client tests not writing aggregated results to files
Root cause: Multi-client tests were parsing aggregated JSON from stdout
but never writing it to the expected output file. This caused:
1. full_result_path to remain None
2. Results not being preserved to aggregated results folder
3. Missing JSON files for multi-client tests (e.g., pubsub tests)

Solution: Write the aggregated results_dict to local_benchmark_output_filename
and set full_result_path appropriately, so the file preservation logic works.

Also removed the None check workaround since it was masking the real issue.
  • Loading branch information
slice4e committed Oct 20, 2025
commit 0748c882b034903ea2b0a03e43bc5db8515d5bd5
25 changes: 13 additions & 12 deletions redis_benchmarks_specification/__runner__/runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -2885,6 +2885,14 @@ def delete_temporary_files(
)
results_dict = json.loads(client_container_stdout)

# Write aggregated results to file so it can be preserved
full_result_path = local_benchmark_output_filename
with open(full_result_path, "w") as json_file:
json.dump(results_dict, json_file, indent=2)
logging.info(
f"Wrote aggregated multi-client results to {full_result_path}"
)

# Validate benchmark metrics
is_valid, validation_error = validate_benchmark_metrics(
results_dict, test_name, benchmark_config, default_metrics
Expand Down Expand Up @@ -3101,19 +3109,12 @@ def delete_temporary_files(
client_aggregated_results_folder,
local_benchmark_output_filename,
)
if full_result_path is not None:
logging.info(
"Preserving local results file {} into {}".format(
full_result_path, dest_fpath
)
)
shutil.copy(full_result_path, dest_fpath)
else:
logging.warning(
"No result file path available to preserve for test {}".format(
local_benchmark_output_filename
)
logging.info(
"Preserving local results file {} into {}".format(
full_result_path, dest_fpath
)
)
shutil.copy(full_result_path, dest_fpath)
overall_result &= test_result

delete_temporary_files(
Expand Down