Description
Background information
When using coll_tuned_dynamic_rules_filename option to tune for a specific system there is no way to set the
alltoall_algorithm_max_requests control variable for the linear_sync alltoall algorithm. It's always 0. The rules file parser does not support reading it from the file, and it is not initialized from the mca control variable.
What version of Open MPI are you using? (e.g., v4.1.6, v5.0.1, git branch name and hash, etc.)
v5.0.3
Describe how Open MPI was installed (e.g., from a source/distribution tarball, from a git clone, from an operating system distribution package, etc.)
git clone
If you are building/installing from a git clone, please copy-n-paste the output from git submodule status.
Please describe the system on which you are running
- Operating system/version: Ubuntu 22.04
- Computer hardware: Intel Xeon
- Network type: Infiniband
Details of the problem
When using coll_tuned_dynamic_rules_filename option to tune for a specific system there is no way to set the
alltoall_algorithm_max_requests control variable for the linear_sync alltoall algorithm. The rules file parser does not support reading it from the file, and it is not initialized from the mca control variable. As a result it is always 0 when using the rules file.
This makes it impossible to reproduce the behavior/performance of the linear_sync algorithm with rules file as compared to when forcing the algorithm via mca control variables, or when using the fixed decision implementations (both of the latter honor the max requests control variable).
Tests using osu_alltoall show that when the max_requests is not set, linear_sync algorithm behaves identically to the linear algorithm. Additionally setting max_requests to 8 on the command line improves alltoall performance over a range of comm and message sizes, but hurts performance over other ranges of comm and message sizes. Therefor, we'd like to be able to set this differently for different comm and message size combos through the dynamic rules file mechanism. At the very least we like to be able to set the max requests control variable on the command line when using the rules file.