Skip to content

[CI] Add new GitHub-based continuous integration#5762

Merged
hugtalbot merged 120 commits intosofa-framework:masterfrom
bakpaul:25_10_add_new_github_based_ci
Dec 8, 2025
Merged

[CI] Add new GitHub-based continuous integration#5762
hugtalbot merged 120 commits intosofa-framework:masterfrom
bakpaul:25_10_add_new_github_based_ci

Conversation

@bakpaul
Copy link
Contributor

@bakpaul bakpaul commented Oct 31, 2025

This PR proposes a CI based on github action for Ubuntu, Fedora and MacOS.
CI PR : sofa-framework/ci#66

First let's talk about our old friendo Windows:

  • Why no Windows support here ? we want here to test the compatibility of our code with the usual package providers of the OS and Windows doesn't have one. The idea is that, once the CI using Pixi ([Pixi][CI] Add pixi support #5252) is available, Windows will be tested there, making conda the supported package provider for Windows.
  • Pixi is not available yet, what about it then ? we will have a transition phase with all windows builder still running on Jenkins but the rest only on github. A branch has been created on the ci repository named jenkins_freeze that'll be used by Jenkins until its inevitable and imminent shutdown.
  • What about the release binaries ? @olivier-roussel worked on making sure the pixi based CI was able to produce some that can be redistributed as the ones created by Jenkins.

Now that this is said, let's dive into the differences with before :

  • ci-depends on :
    1. The main difference is that before a merge of the PR on the other repository was made with the main branch of this repository, blocking sometimes the CI. Now the working branch is directly used.
    2. The second difference is that now a ci-depends-on can be done in the ci repository itself. This allow to test any changes made on the ci scripts before merging them. But this will require to increase the security on the PR, now for all outsiders, we will be responsible of authorizing the launch of the CI, which makes more sens
  • tags that we put in messages to launch the ci [xxx][xxx]:
    1. Launching ci through messages is not supported anymore. This was handy when we needed to log into Jenking to relaunch a job. Now the Github interface is enough, the [ci-build] doesn't exist anymore
    2. We can still configure two things using those tags : [force-full-build][with-all-tests]. The only difference is now that those tags have to be bu in the PR description, same as [ci-depends-on]
  • labels: Now if the 'to review' label is not put, the the ci doesn't launch
  • dashboard: Now that all information are available directly in the Github interface, we've decided to remove the (in)famous sofa-framework.org/dash. The changes needed to keep it were not worth the added value it has w/r to the Github interface.
  • logs: The Jenkins interface, even if a bit broken, was handy for tests logs. Now we don't have this anymore. Plus, were used to having a summary of the tests results directly on the github check (Build OK. FIXME: 0 unit, 0 scene, 0 regression), as now those checks are github based, it is not possible anymore the modify them, at least not easilly.
    1. To ease the logs reading and exploration, a 'Summarize results' step is always run at the end of the build-and-tests action that'll summarize the build results, and all tests results in a quick and handy way, even displaying the logs of the failed tests (see https://github.com/bakpaul/sofa/actions/runs/18944317329?pr=42).
    2. All logs are published as artifacts (configure & build logs and tests logs), you can also see them in the above link at the bottom of the main page, and hey are recalled in the Summarize results step when failed.
  • cache gestion: The MacOS builders are usually the fastest ones, this is because it is the only one having a working ccache configuration. On Windows ccache was faulty, and it has never been activated for docker images and I don't know why. Now ccache is used even on docker images, the cache are saved on the host system and shared amongst runs, which makes the run go way faster.
  • cleaning: All runner are rebooted at around 1 a.m., this will tigger two things :
    1. A clean of working directory (which also mean that each run build files are kept on the builder during the day for debug purposes if neede)
    2. A clear of ccache on Linux
    3. The reboot action can be run by hand, so any of us can reboot a builder if it is faulty during the day
  • github related: To keep track of all the build during the day and filter who can run jobs on our runners, a pre-job script is run every time a job is triggered and before it run. This is done using an environment variable called ACTIONS_RUNNER_HOOK_JOB_STARTED (same exists for cleanup bu we don't use it). This script is present here : https://github.com/bakpaul/ci/tree/jenkins_gha_migration/scripts/github-hookups and every builder has a clone of the master branch of the ci repository to access those script. This repository is updated ever 10 seconds, so any modification of those script pushed to the master branch of the ci repo will be effective directly on the builders.

Finally, a script to setup a builder on either macos or ubuntu is proposed here : https://github.com/bakpaul/ci/blob/jenkins_gha_migration/setup/github-runner/unix.sh

Last questions that you might have are :

  • Why keep the old behavior of ci script in a repository instead of coding everything in the action ? Well because on github, a modification to the action cannot be tested before being merged inside the master branch. So, to still be able to test the changes in the logic, we wanted to keep the yml file small.
  • What about the plugins CI ? Again, as for windows, the conda based ci should be the one responsible of generating the "nightly" packages used by the plugins CI. But until then, one builder of each os will be kept on jenking just for this.

And finally, to have no scene tests errors, we will need to merge this PR : #5752


By submitting this pull request, I acknowledge that
I have read, understand, and agree SOFA Developer Certificate of Origin (DCO).


Reviewers will merge this pull-request only if

  • it builds with SUCCESS for all platforms on the CI.
  • it does not generate new warnings.
  • it does not generate new unit test failures.
  • it does not generate new scene test failures.
  • it does not break API compatibility.
  • it is more than 1 week old (or has fast-merge label).

bakpaul and others added 30 commits April 22, 2025 15:21
* Push space to creatye PR

* Finished clone sofa and ci stage

* Rename action

* fix some typos in filter script
* Commit to open a PR

* revert env modification

* Move steps around + add os configuration

* Configure step should be working

* Move comment around

* Step build
* Use step outputs to send variables in call workflow

* add BUILDER_OS environment variable

* add check for all tests in PR body

* remove useless secret

* udpate PR python script

* add missing runs-on: ubuntu-latest

* Fix build action/ ci-depends-on

* remove runs-on

* update python script

* clean python script

* fix export of booleans

* Fix action

---------

Co-authored-by: Paul Baksic <paul.baksic@outlook.fr>
* Update checkPRInfoBeforeBuild.py

* Apply suggestions from code review

Co-authored-by: Paul Baksic <30337881+bakpaul@users.noreply.github.com>

* Apply suggestions from code review

* Apply suggestions from code review

* Test

* Use this script

* fix

* test fromJson

* add debug

* Final fix

* revert change

* Refacto display action

* Fix filter

---------

Co-authored-by: Paul Baksic <30337881+bakpaul@users.noreply.github.com>
Co-authored-by: Paul Baksic <paul.baksic@outlook.fr>
* TO BE REVERTED BEFORE MERGE

* Fix syntax

* Fix git handle

* fix action location

* fix typo

* Fix call to load vm env
* Isolate nightly build to allow for matrix only in nightly cases

* Check that SOFA branch name is either master or vXX.XX

* Attempt to detect existing running builds on PR

* Fix check on ci-depends-on

* fix indent and add generic condition for nightly build

* Another fix for ci-depends-on

* Another fix for ci-depends-on in json

* Display info on PR comment

* Display info on PR comment

* Display info on PR comment
* Fix trigger through comments

* remove check when action is already triggered

* remove test comment
@bakpaul bakpaul added the pr: status to review To notify reviewers to review this pull-request label Oct 31, 2025
@hugtalbot
Copy link
Contributor

🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉
🤖
🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉

@hugtalbot hugtalbot added the pr: highlighted in next release Highlight this contribution in the notes of the upcoming release label Oct 31, 2025
@bakpaul bakpaul added the topic for next dev-meeting PR to be discussed in sofa-dev meeting label Oct 31, 2025
@hugtalbot hugtalbot changed the title [CI] Add new GitHub based ci [CI] Add new GitHub-based continuous integration Nov 4, 2025
@hugtalbot hugtalbot removed the topic for next dev-meeting PR to be discussed in sofa-dev meeting label Nov 13, 2025
@hugtalbot
Copy link
Contributor

Let's notify @sofa-framework/reviewers
This PR waits until early December then will be merged if no feedback is given

Cheers 🎉

@hugtalbot hugtalbot added this to the v25.12 milestone Dec 4, 2025
@bakpaul bakpaul added pr: status ready Approved a pull-request, ready to be squashed and removed pr: status to review To notify reviewers to review this pull-request labels Dec 4, 2025
@hugtalbot hugtalbot merged commit 845b0f0 into sofa-framework:master Dec 8, 2025
8 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr: highlighted in next release Highlight this contribution in the notes of the upcoming release pr: new feature Implement a new feature pr: project-ci-infrastructure pr: status ready Approved a pull-request, ready to be squashed

Projects

No open projects

Development

Successfully merging this pull request may close these issues.

2 participants