[FEA] Scatter and Gather calls in cuml comms

[cjnolet]

There are a few places in cuML where we are using UCX endpoints directly to perform scatter and gather calls. We originally kept the API minimal, since users could compose their own scatter and gather calls out of the isend and irecv calls.

However, this can be error prone and we're already finding small subtle places where we diverged from the MPI spec and this caused unexpected behaviors. While I don't think we should claim to adhere strictly to the MPI spec, I do think that including the scatter and gather calls in the comms would alleviate some of this, since they are common communications primitives.

[github-actions]

This issue has been marked rotten due to no recent activity in the past 90d. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

[github-actions]

This issue has been marked stale due to no recent activity in the past 30d. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be marked rotten if there is no activity in the next 60d.