Skip to content

split testing libraries from run libraries in dependencies.yaml #5447

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jayavenkatesh19 wants to merge 2 commits into
base: main
Choose a base branch
from
Open

split testing libraries from run libraries in dependencies.yaml #5447

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a36ee26
split testing libraries from run libraries
jayavenkatesh19 Feb 27, 2026
8dea84c
Merge branch 'main' into split-test-libraries
jayavenkatesh19 Mar 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 22 additions & 2 deletions dependencies.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ files:
- python_build_wheel
- python_run_cugraph
- test_notebook
- run_python_common
- test_python_common
- test_python_cugraph
- test_python_pylibcugraph
Expand Down Expand Up @@ -62,12 +63,24 @@ files:
- depends_on_libcugraph
- depends_on_libcugraph_etl
- depends_on_libcugraph_tests
run_notebooks:
output: none
includes:
- cuda_version
- py_version
- test_notebook
- run_python_common
- test_python_cugraph
- depends_on_libcugraph
- depends_on_pylibcugraph
- depends_on_cugraph
Copy link
Member

@jameslamb jameslamb Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do like creating a new top-level list called just run_notebooks, but think it's confusing to see anything named test_* showing up in this list, especially to see something called test_notebook showing up in run_notebooks when there is also a test_notebooks.

I have an alternative proposal that I think might make this a bit easier to maintain and understand, let me know what you think:

  1. leave test_python_common completely unchanged
  2. create a new list under dependencies: called run_notebooks. Add to that list only packages that are needed for the notebooks and aren't already runtime dependencies of cugraph, pylibcugraph, and libcugraph. Use YAML anchors to reference other things already in dependencies.yaml. Add a code comment above it explaining its purpose.
  3. make this top-level run_notebooks list here contain cuda_version, py_version, run_notebooks, and the depends_on_*cugraph groups
  4. pass --file-key test_notebooks --file-key run_notebooks here https://github.com/rapidsai/docker/blob/2637556074567056aa69a7cb1850ac844ef89748/context/notebooks.sh#L34 (multiple --file-key are allowed)

That way, we could fine-tune this to exactly the libraries that are needed and nothing else, and the purpose will be clear.

test_notebooks:
output: none
includes:
- cuda_version
- py_version
- test_notebook
- run_python_common
- test_python_common
- test_python_cugraph
- depends_on_libcugraph
Expand All @@ -80,6 +93,7 @@ files:
- depends_on_cudf
- depends_on_pytorch
- py_version
- run_python_common
- test_python_common
- test_python_cugraph
- test_python_pylibcugraph
Expand Down Expand Up @@ -134,6 +148,7 @@ files:
table: project.optional-dependencies
key: test
includes:
- run_python_common
- test_python_common
- test_python_cugraph
py_build_libcugraph:
Expand Down Expand Up @@ -202,6 +217,7 @@ files:
key: test
includes:
- depends_on_cudf
- run_python_common
- test_python_common
- test_python_pylibcugraph
channels:
Expand Down Expand Up @@ -392,16 +408,20 @@ dependencies:
- certifi
- ipython
- notebook>=0.5.0
test_python_common:
run_python_common:
common:
- output_types: [conda, pyproject, requirements]
packages:
- &pandas pandas
- scipy
test_python_common:
common:
- output_types: [conda, pyproject, requirements]
packages:
- pytest<9.0.0
- pytest-benchmark
- pytest-cov
- pytest-xdist
- scipy
test_python_cugraph:
common:
- output_types: [conda, pyproject, requirements]
Expand Down
Loading