Using all_neighbors for mutual reachability

[jinsolp]

Using all_neighbors mutual reachability feature to work towards deprecating cuvs::neighbors::detail::reachability::mutual_reachability_graph (related issue: #982).

Also, changes build_linkage to use int64_t, because all_neighbors::build only supports int64_t types, and we will change HDBSCAN to use int64_t types in cuML as well.

This PR will break cuML (which will be solved by rapidsai/cuml#7104)

Waiting for #1016 to be merged.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[beckernick]

Excited to see how HDBSCAN scales!

[cjnolet]

We should probably add some docs for this new option and highlight how it differs from the distance_params.

[jinsolp]

I'm kind of confused here because the build_linkage function already had two options (mutual reach and distance) for the params. We just have two different APIs now (one for host data and one for device data) 👀 The real new thing here is adding cuvs::neighbors::all_neighbors::all_neighbors_params to mutual_reachability_params

[jinsolp]

@cjnolet @tarang-jain I've added what all_neighbors_params is for in struct mutual_reachability_params. I didn't add details because I thought it makes sense for the users to look into the docs of all_neighbors_params itself for details. Should we be adding the details here too?

[cjnolet]

I didn't add details because I thought it makes sense for the users to look into the docs of all_neighbors_params itself for details

We definitely should document the mutual reachabilty feature in the linkage function itself and not expect users to look for themselves. It doesn't need to be exhaustive, but the docs for every argument should summarize the intention and purpose fo the argument.

[tarang-jain]

LGTM! I yet have to read the new documentations for the APIs in this PR. But otherwise just pointing out some small changes.