Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce CI costs #5332

Open
6 of 10 tasks
emilk opened this issue Feb 28, 2024 · 4 comments
Open
6 of 10 tasks

Reduce CI costs #5332

emilk opened this issue Feb 28, 2024 · 4 comments
Labels
🧑‍💻 dev experience developer experience (excluding CI) 🚢 CI
Milestone
Spring Cleaning

Comments

@emilk
Copy link
Member

emilk commented Feb 28, 2024 •
edited
Loading

GitHub actions is costing us more and more each month. Time to rein it it a bit.

We should consider:

  • Move jobs from PR to main
  • Move jobs from main to nightly
  • Only run jobs if needed (e.g. ignore Rust jobs when on docs changes)
  • Adjust what is run on cargo checks (we have a python script that governs that)
  • Look for redundancies (are we building the web viewer multiple times per PR? 🤷 )
  • Consider not running the full CI on each commit
  • Use cheaper (but slower) GitHub runners for some jobs
  • Move away from GitHub actions
  • Compile Wasm in debug mode (skip wasm-opt)?
  • Skip cargo test on each PR (just on main)

The "Billable time" column here should help us to focus:
@emilk emilk added 🧑‍💻 dev experience developer experience (excluding CI) 🚢 CI labels Feb 28, 2024
@Wumpf Wumpf mentioned this issue Feb 28, 2024
Merged
5 tasks
Wumpf added a commit that referenced this issue Feb 28, 2024
@Wumpf
Run C++/rust CI only on C++/rust changes (#5333)
3f881ab 
### What

* Part of #5332

C++ had some of this already but this PR makes it more aggressive:
* don't run _any_ cpp checks if no cpp file was touched
* move C++ formatting checks to the cpp check workflow
* split out rust checks and for PRs only run them when rust or toml
files were touched

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using newly built examples:
[app.rerun.io](https://app.rerun.io/pr/5333/index.html)
* Using examples from latest `main` build:
[app.rerun.io](https://app.rerun.io/pr/5333/index.html?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[app.rerun.io](https://app.rerun.io/pr/5333/index.html?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!

- [PR Build Summary](https://build.rerun.io/pr/5333)
- [Docs
preview](https://rerun.io/preview/ee71a7da27fbc49f343046003bcdb75e91ef5f9a/docs)
<!--DOCS-PREVIEW-->
- [Examples
preview](https://rerun.io/preview/ee71a7da27fbc49f343046003bcdb75e91ef5f9a/examples)
<!--EXAMPLES-PREVIEW-->
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)
@teh-cmc
Copy link
Member

teh-cmc commented Feb 28, 2024

Any feature permutation will wipe the cargo cache, which means cargo will rebuild the entire workspace from scratch any time that happens.
I wouldn't be surprised if we do so at least ten times for each CI run right now (every different clippy/build/test/doc invocations, every different SDKs, etc), probably even more.

At the very least, the PR CI should only build the most important feature permutations, and keep everything else for the main branch.

@teh-cmc
Copy link
Member

teh-cmc commented Feb 28, 2024

We should also try cheaper runners, which would also cover our arm64 needs.

@chengguizi
Copy link

Also, seems that the C++ part always build examples and docs

rerun/CMakeLists.txt

Lines 158 to 160 in 1643db8

add_subdirectory(examples/cpp)
add_subdirectory(tests/cpp)
add_subdirectory(docs/code-examples)

would be great if that can be a cmake option. So building CI/CD as someone using rerun saves time

@Wumpf
Copy link
Member

Wumpf commented Mar 1, 2024

@chengguizi this means that a single CMake's configure always knows of all the targets, but we're not necessarily building all the targets on every CI step

This was referenced Apr 10, 2024
Closed
Open
@emilk emilk assigned emilk and unassigned emilk Apr 10, 2024
@emilk emilk mentioned this issue Apr 12, 2024
Merged
5 tasks
emilk added a commit that referenced this issue Apr 12, 2024
@emilk
Move many CI jobs from main to nightly (#5937)
78d46da 
### What
* A huge part of #5332

This moves _a lot_ of jobs from `main` to nightly:

* Building and testing wheels, `rerun_c` and `rerun-cli` for all
platforms (keeping just linux-x86 on `main`)
* Doing a pre-release
* Benchmarks
* Clean build

This should reduce the billable time on `main` significantly.

I've mostly just moved the jobs (leaving a few behind) and changing
`CONCURRENCY:` to `nightly` everywhere.

### Checklist
* [x] I have read and agree to [Contributor
Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and
the [Code of
Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)
* [x] I've included a screenshot or gif (if applicable)
* [x] I have tested the web demo (if applicable):
* Using examples from latest `main` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/5937?manifest_url=https://app.rerun.io/version/main/examples_manifest.json)
* Using full set of examples from `nightly` build:
[rerun.io/viewer](https://rerun.io/viewer/pr/5937?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json)
* [x] The PR title and labels are set such as to maximize their
usefulness for the next release's CHANGELOG
* [x] If applicable, add a new check to the [release
checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)!

- [PR Build Summary](https://build.rerun.io/pr/5937)
- [Recent benchmark results](https://build.rerun.io/graphs/crates.html)
- [Wasm size tracking](https://build.rerun.io/graphs/sizes.html)
@emilk emilk removed their assignment Apr 12, 2024
@emilk emilk added this to the Spring Cleaning milestone Apr 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
🧑‍💻 dev experience developer experience (excluding CI) 🚢 CI
Projects
None yet
Milestone
Spring Cleaning
Development

No branches or pull requests
4 participants
@emilk @Wumpf @teh-cmc @chengguizi