Skip to content

hydra: avoid duplicate proxies in spawn#7712

Open
hzhou wants to merge 1 commit intopmodels:mainfrom
hzhou:2602_hydra_spawn
Open

hydra: avoid duplicate proxies in spawn#7712
hzhou wants to merge 1 commit intopmodels:mainfrom
hzhou:2602_hydra_spawn

Conversation

@hzhou
Copy link
Contributor

@hzhou hzhou commented Feb 3, 2026

Pull Request Description

When user spawn with a host list in info hints, hydra creates new user_node_list, which results in launching new proxies. We only need the separate user_node_list for figuring out where to create the new processes, i.e. rankmap; but we should merge to the main node list to reuse the proxies.

For example, the following code will create multiple proxies even though all processes are launched on the local host:

    MPI_Info_create(&info);
    MPI_Info_set(info, "host", hostname);
    MPI_Info_set(info, "map-by", "ppr:1:node");
    MPI_Info_set(info, "bind_to", "core");

    for (int i = 0; i < 5; ++i)
    {
        MPI_Comm_spawn("./TestSpawn", argv, 1, info, 0, MPI_COMM_SELF, &null_comm, error_codes);
    }

[skip warnings]

Author Checklist

  • Provide Description
    Particularly focus on why, not what. Reference background, issues, test failures, xfail entries, etc.
  • Commits Follow Good Practice
    Commits are self-contained and do not do two things at once.
    Commit message is of the form: module: short description
    Commit message explains what's in the commit.
  • Passes All Tests
    Whitespace checker. Warnings test. Additional tests via comments.
  • Contribution Agreement
    For non-Argonne authors, check contribution agreement.
    If necessary, request an explicit comment from your companies PR approval manager.

When user spawn with a host list in info hints, hydra creates new
user_node_list, which results in launching new proxies. We only need the
separate user_node_list for figuring out where to create the new
processes, i.e. rankmap; but we should merge to the main node list to
reuse the proxies.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant