Experiment request

[am009]

We are trying to improve AFL++'s performance on proj4_proj_crs_to_crs_fuzzer. We have created two improved versions that have been initially verified on our local experiment, and we would like to further evaluate them on more benchmarks. Additionally, since the original AFL++ version is quite old, we have also included an aflplusplus_recent fuzzer.

Will the generated report be merged with results from other existing results? It seems there is a merge_with_nonprivate option, which is set to true in the YAML file. The command is probably:

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-07-06-aflpp-hfdict --fuzzers aflplusplus_hfdict_v1 aflplusplus_hfdict_v2 aflplusplus_recent

[vanhauser-thc]

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-07-06-aflpp-hfdict --fuzzers aflplusplus_hfdict_v1 aflplusplus_hfdict_v2 aflplusplus_recent

[vanhauser-thc]

sadly you are running into the bug that no coverage data is being generated.
once the experiment is finished you need to download all data yourself and analyse by hand :-(

https://storage.googleapis.com/fuzzbench-data/index.html?prefix=2025-07-06-aflpp-hfdict/

[am009]

sadly you are running into the bug that no coverage data is being generated. once the experiment is finished you need to download all data yourself and analyse by hand :-(

https://storage.googleapis.com/fuzzbench-data/index.html?prefix=2025-07-06-aflpp-hfdict/

I briefly looked through the files and it seems like there are no issues. Do you know how to analyze this issue? Or is there any more detailed error information available?

[vanhauser-thc]

I dont know what the bug on fuzzbench is. but the data is there or is being generated, so once it is done - download it and do coverage analysis on the corpus data. (or you just extract the latest fuzzer_stats file save from each run, the bitmap_cvg stat shows the coverage reached.)

[vanhauser-thc]

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-07-21-aflpp-hfdict --fuzzers aflplusplus_recent aflplusplus_hfdict_v2

[vanhauser-thc]

I have to restart the experiment, some have only 3 completed runs :-(

[vanhauser-thc]

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-07-24-aflpp-hfdict --fuzzers aflplusplus_recent aflplusplus_hfdict_v2

[am009]

The report seems a bit strange; it appears to be missing many fuzzers. What should I do next

https://storage.googleapis.com/www.fuzzbench.com/reports/experimental/2025-07-24-aflpp-hfdict/index.html

[am009]

Add a new variant aflplusplus_hfdict_v1_1.

[vanhauser-thc]

/gcbrun run_experiment.py -a --experiment-config /opt/fuzzbench/service/experiment-config.yaml --experiment-name 2025-08-15-aflpp-hfdict --fuzzers aflplusplus_recent aflplusplus_hfdict_v2 aflplusplus_hfdict_v1_1