Update RAPIDS Python packages to use rapids-build-backend

[vyasr]

rapids-build-backend is a wrapper around standard backends like scikit-build-core and setuptools that handles some of the standard issues that we face for RAPIDS packages (CUDA versioning, alpha versions, etc). Substituting it into existing RAPIDS Python packages should be fairly painless, but will require some careful testing to verify that nothing is broken.

The backend need not be updated in lockstep across all of RAPIDS for this to work, and merging PRs in any order should be generally safe. However, since there are cases like unified devcontainers where underlying build commands may need an update, it would be best to try and test at least a couple of core packages together to verify that everything works as expected.

Changes to rapids-build-backend and rapids-dependency-file-generator

Preview Give feedback

1. Automatically write GIT_COMMIT file (Automatically figure out where to write GIT_COMMIT rapids-build-backend#30)
2. Add disable-cuda option (Add disable-cuda option rapids-build-backend#32)
3. Warn if dependencies.yaml is missing (warn if dependencies file does not exist rapids-build-backend#33)
4. Release version 0.2.0 (Version 0.2.0 rapids-build-backend#34)
5. Prefix -C options with rapidsai. (Prefix and filter command line options with rapidsai. rapids-build-backend#37)
6. support projects with non-trivial setup.py ([RFC] handling CUDA-suffixed dependencies needed at setuptools.setup() time rapids-build-backend#39)
7. set up publishing to public PyPI (Enable release wheels publish to pypi rapids-build-backend#47)

Migrations

Preview Give feedback

1. cucim (use rapids-build-backend cucim#736)
2. cudf (Use rapids-build-backend. cudf#15245)
3. cugraph (Use rapids-build-backend cugraph#4393)
4. cugraph-ops (https://github.com/rapidsai/cugraph-ops/pull/652)
5. cuml (Use rapids-build-backend cuml#5804)
6. cuopt (deferred until after RAPIDS 24.08, https://github.com/rapidsai/cuopt/issues/1879)
7. cuspatial (use rapids-build-backend cuspatial#1393)
8. cuvs (Use rapids-build-backend cuvs#145)
9. cuxfilter (use rapids-build-backend cuxfilter#600)
10. dask-cuda (use rapids-build-backend dask-cuda#1343)
11. kvikio (use rapids-build-backend kvikio#389)
12. pynvjitlink (use rapids-build-backend pynvjitlink#88)
13. raft (Use rapids-build-backend raft#2331, ensure update-version.sh preserves alpha spec, add tests on version constants raft#2344, Change --config-setting to --config-settings raft#2342)
14. rmm (Use rapids-build-backend. rmm#1502)
15. ucx-wheels (use rapids-build-backend ucx-wheels#9)
16. ucx-py (use rapids-build-backend ucx-py#1044, make pip source installs a bit easier ucx-py#1048)
17. ucxx (use rapids-build-backend ucxx#234)
18. wholegraph (use rapids-build-backend wholegraph#181)

Other

Preview Give feedback

1. Hook to verify alpha spec in dependencies.yaml (Add verify-alpha-spec hook pre-commit-hooks#28)
2. Any necessary rapidsai/devcontainer adjustments (support rapids-build-backend devcontainers#325)
3. replace "commit-file" with "commit-files" (or just remove that configuration) in projects that were updated before Automatically figure out where to write GIT_COMMIT rapids-build-backend#30
4. remove .gitattributes files left behind from when projects used versioneer (Update RAPIDS Python packages to use rapids-build-backend #31 (comment))

dependencies.yaml updates (for devcontainers / DLFW)

Preview Give feedback

1. devcontainers (Explicitly select for CUDA-suffixed dependencies in rapids-make-pip-dependencies devcontainers#365)
2. cucim (not needed... only CUDA-suffixed dependency is cupy-cuda{nn}x, and we want to keep that suffixed)
3. cudf (split up CUDA-suffixed dependencies in dependencies.yaml cudf#16183)
4. cugraph ( split up CUDA-suffixed dependencies in dependencies.yaml cugraph#4552)
5. cugraph-ops (not needed... no CUDA-suffixed dependencies)
6. cugraph-gnn (split up CUDA-suffixed dependencies in dependencies.yaml cugraph-gnn#16)
7. cuml (split up CUDA-suffixed dependencies in dependencies.yaml cuml#5974)
8. cuspatial (split up CUDA-suffixed dependencies in dependencies.yaml cuspatial#1414)
9. cuvs (split up CUDA-suffixed dependencies in dependencies.yaml cuvs#247)
10. cuxfilter (split up CUDA-suffixed dependencies in dependencies.yaml cuxfilter#609)
11. dask-cuda (split up CUDA-suffixed dependencies in dependencies.yaml dask-cuda#1364, follow-up in consolidate cuda_suffixed=false blocks in dependencies.yaml, fix update-version.sh dask-cuda#1367)
12. kvikio ( split up CUDA-suffixed dependencies in dependencies.yaml kvikio#411)
13. pynvjitlink (not needed... no CUDA-suffixed dependencies)
14. raft (split up CUDA-suffixed dependencies in dependencies.yaml raft#2388)
15. rmm (split up CUDA-suffixed dependencies in dependencies.yaml rmm#1627)
16. ucx-wheels (not needed ... no CUDA-suffixed dependencies, no dependencies.yaml)
17. ucx-py (split up CUDA-suffixed dependencies in dependencies.yaml ucx-py#1057)
18. ucxx (split up CUDA-suffixed dependencies in dependencies.yaml ucxx#251)
19. wholegraph (not needed ... no CUDA-suffixed dependencies, no dependencies.yaml)
20. forward-merge all these changes into branch-24.10
21. DLFW 24.08 build (RAPIDS/dlfw!158 in private repo)