RAFT: CUDA 12 Conda Packages

[jakirkham]

Pushing for CUDA 12 Conda package builds of libraries for RAPIDS 23.04. This issue is for tracking that work.

Requirements (please recheck):

Developer tasks

Preview Give feedback

1. I've confirmed the list below is correct (or updated if not)

Toolkit dependencies from Conda

Preview Give feedback

1. Adding cuda-cudart conda-forge/staged-recipes#21723
   review-requested staged-recipes +2
2. Adding cuda-nvcc recipe conda-forge/staged-recipes#21350
   +0
3. Adding cuda-profiler-api recipes conda-forge/staged-recipes#22125
   +0
4. Adding cuda-nvtx recipe conda-forge/staged-recipes#22052
   +0
5. Add libcublas recipe conda-forge/staged-recipes#21901
   review-requested staged-recipes +2
6. Add libcurand recipe conda-forge/staged-recipes#21916
   +0
7. Add libcusolver, libcusparse, and libnvjitlink recipes conda-forge/staged-recipes#21924
   review-requested staged-recipes +2
8. Adding cuda-cccl recipe conda-forge/staged-recipes#21953
   review-requested staged-recipes +2

RAPIDS Conda dependencies

Preview Give feedback

1. RMM: CUDA 12 Conda Packages rmm#1207
   6 of 6
   +0
   

Other dependencies from Conda using CUDA

Preview Give feedback

1. NCCL: Support CUDA 12 conda-forge/nccl-feedstock#82
   +0
2. UCX: Support CUDA 12 conda-forge/ucx-split-feedstock#116
   +0

Deprecated dependencies from Conda using CUDA ( please see PR #1202 )

Preview Give feedback

1. Faiss: Support CUDA 12 conda-forge/faiss-split-feedstock#65
   +0

Test only dependencies from Conda using CUDA

Preview Give feedback

1. CuPy: Support CUDA 12 conda-forge/cupy-feedstock#139
   +0

[bdice]

cuda-cccl is not a dependency for any RAPIDS packages. RAPIDS usually uses newer (and sometimes patched) versions of CCCL fetched by rapids-cmake, and won't be needing the cuda-cccl packages.

[bdice]

All the math libraries will require -dev packages. e.g. libcublas-dev as well as libcublas. We list the -dev packages as runtime requirements in libraft-headers so that other packages can build against it.

[bdice]

Also needs nccl and ucx with CUDA support listed somewhere.

[jakirkham]

For now will leave cuda-cccl just so we know it is used in some form. How exactly we handled that we can follow up after the general discussion around it has settled.

Good to know about -dev packages being used at runtime. The -dev packages are produced out of a split package that also produces the dynamic & static library packages. So knowing one of them is needed is sufficient. We can take a closer look when it comes time to rebuild.

Thanks for reminding me 🙏 Had forgotten these 🤦‍♂️ They are now in the other dependencies list.

[cjnolet]

I agree w/ @bdice, we don't explicitly depend on anything from cuda-cccl currently. The faiss dependency will also be going away shortly (definitely in 23.04) w/ this pr

[jakirkham]

Thanks Corey! 🙏

Since cuda-nvcc is going to depend on cuda-cccl, am going to leave it.

Noted that faiss is going away, but have left that item in the list for now. Believe merging that PR should check that box, but we can do this manually if that ends up being needed.

[cjnolet]

@jakirkham this LGTM. Just looked back through the dependencies to verify. cupy is only a test dependency and everything else is covered.

[bdice]

Working on builds with the nvidia channel here: #1353

[jakirkham]

Current work using nvidia channel packages in PR ( #1388 )