Skip to content

Migrate conda packaging to rattler-build (v1 recipe format) for GitHub Actions CI - #3031

Open
ekomarova wants to merge 7 commits into
IntelPython:masterfrom
ekomarova:rattler-build-migration
Open

Migrate conda packaging to rattler-build (v1 recipe format) for GitHub Actions CI#3031
ekomarova wants to merge 7 commits into
IntelPython:masterfrom
ekomarova:rattler-build-migration

Conversation

@ekomarova

@ekomarova ekomarova commented Aug 17, 2026

Copy link
Copy Markdown
Member

Add rattler-build support for GitHub Actions CI

Summary

This PR adds rattler-build as the package-building tool for the Conda package GitHub Actions workflow, alongside a new recipe.yaml (v1 recipe schema) generated from the existing meta.yaml.

meta.yaml is left untouched and fully functional. This PR only changes what runs on GitHub Actions; it does not retire or replace the conda-build path.

What changed

  • .github/workflows/conda-package.yml: build steps now call rattler-build instead of conda-build

  • conda-recipe/recipe.yaml: new file, the rattler-build (v1 schema) equivalent of meta.yaml. A handful of things had to be restructured because recipe.yaml must always be valid plain YAML (no Jinja control-flow blocks)

  • environments/build_conda_pkg.yml: now installs rattler-build instead of conda-build/conda-verify

  • conda-recipe/bld.bat: unchanged, still the source file for conda-build; a CI step copies it to build.bat on Windows runners since that's the filename rattler-build expects

Why these specific changes were needed

  • CONDA_BLD path — rattler-build outputs to output/<platform>/ relative to the working directory by default, unlike conda-build's $CONDA_PREFIX/conda-bld/

  • Compute version from git describe step — conda-build runs git describe internally before rendering to populate GIT_DESCRIBE_TAG/GIT_DESCRIBE_NUMBER. rattler-build has no equivalent, so recipe.yaml now reads these via env.get(...), and we compute + export them ourselves beforehand.

  • Prepare build.bat for rattler-build step — rattler-build expects build.bat (symmetric with build.sh), while conda-build (still used in non-GitHub CI) only recognizes bld.bat. Rather than duplicating the file, bld.bat stays the single source and gets copied to build.bat at CI time on Windows.

  • RATTLER_CHANNELS array — channels-list includes --override-channels, which rattler-build's CLI doesn't recognize, so it's stripped before use. Expanded into a bash array (instead of a bare $VAR) so each -c <channel> reaches rattler-build as a separate argument — also fixes a shellcheck SC2086 warning.

  • Removed conda-verify=3.4.2 — it's a conda-build-specific post-build checker; rattler-build validates internally, so it's no longer needed.

  • Removed the two guard blocks and the pyproject.toml loop in recipe.yamlrecipe.yaml must always be valid plain YAML, so Jinja control-flow ({% if %}/{% for %}) isn't supported. The version guard checks are dropped (an invalid override still fails to build, just via a less friendly solver error instead of a custom message), and the dependency list previously generated from pyproject.toml's [build-system] requires is now static — it needs manual updates if those requirements change.

  • Added tests: - script: ... block — v1 schema requires an explicit test type; the old test: requires: block installed test deps without running anything, which isn't valid anymore. Added a minimal import/version smoke test

  • Replace miniconda with rattler-build action in the build job — rattler-build is a standalone binary with no conda/Python dependency, so building conda packages with it doesn't need a full conda distribution on the runner. The installation takes about 1s

@coveralls

coveralls commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator

Coverage Status

coverage: 78.459% (+0.002%) from 78.457% — ekomarova:rattler-build-migration into IntelPython:master

@ekomarova ekomarova self-assigned this Aug 18, 2026
@ekomarova
ekomarova force-pushed the rattler-build-migration branch from 3991bb9 to ee3ec67 Compare August 18, 2026 14:29
@ekomarova
ekomarova force-pushed the rattler-build-migration branch from b9750b9 to b2a68e2 Compare August 18, 2026 17:12
@ekomarova

ekomarova commented Aug 18, 2026

Copy link
Copy Markdown
Member Author

@antonwolfy @ndgrigorian It looks like I've managed to stabilize GitHub CI, so please take a look at it when you have time. rattler-build wins in terms of build speed and dependency resolution

I would also consider getting rid of installing miniconda for conda_package test pipeline, since rattler-build is standalone and installed fairly quickly. According to my observations, all that miniconda is used for is indexing, searching, installation, and all this supports by rattler and pixi, so migrating the test part to a faster rattler/pixi also makes sense. But this is a separate task that I would not make part of this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants