Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI improvements #173

Merged
merged 9 commits into from
Feb 7, 2024
Merged

CI improvements #173

merged 9 commits into from
Feb 7, 2024

Conversation

ThibsG
Copy link
Contributor

@ThibsG ThibsG commented Feb 5, 2024

  • remove large files (> 40 MB) before saving cache
  • stop compiling for running tests on binaries (0 tests to pass, but still 4 mins to compile)
  • unify binary compilations in a single line by using --bins

CI total time

before: 41 mins
after: 31 mins

  • stop doing release jobs at every push. Now a nightly pipeline will do the release jobs for develop (aka latest_build release packages). Tags still trigger a release build

CI total time

before: 31 mins
after: 10 mins

@ThibsG ThibsG changed the base branch from main to develop February 5, 2024 13:47
@ThibsG ThibsG force-pushed the ci_optimizations branch 4 times, most recently from 99e7ffb to 2894fc9 Compare February 5, 2024 15:10
Copy link
Contributor

@JosePisco JosePisco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job! It might come in very handy!

Also, if I understand the binary build, we don't strip the binary. We can try to set the strip flag to true, it lightens the produced binary by removing all unnecessary symbols and debug informations.

[profile.release]
strip = "debuginfo"

Furthermore, I think we can activate LTOs (Link Time Optimizations) in rust to lighten the binary even more.

Quickly, LTOs allow the compiler to keep functions information aside and the linker can optimize even more thanks to those informations becaue he's got all final compiled files to deal with. In the cases that some short functions are in libraries (which might often be the case in all the externals crates we use), the compiler will never inline them because they need to be used by the binary, but the linker can see it when it's got all compiled files, and it can inline it in the end.
It's useful to get rid of some more symbols but we need to know that it actually slows down the compile process by some important factor.

@ThibsG
Copy link
Contributor Author

ThibsG commented Feb 6, 2024

Great job! It might come in very handy!

Also, if I understand the binary build, we don't strip the binary. We can try to set the strip flag to true, it lightens the produced binary by removing all unnecessary symbols and debug informations.

[profile.release]
strip = "debuginfo"

Furthermore, I think we can activate LTOs (Link Time Optimizations) in rust to lighten the binary even more.

Quickly, LTOs allow the compiler to keep functions information aside and the linker can optimize even more thanks to those informations becaue he's got all final compiled files to deal with. In the cases that some short functions are in libraries (which might often be the case in all the externals crates we use), the compiler will never inline them because they need to be used by the binary, but the linker can see it when it's got all compiled files, and it can inline it in the end. It's useful to get rid of some more symbols but we need to know that it actually slows down the compile process by some important factor.

Hey thanks, you are totally right 😉 The binaries are already stripped and LTO is already activated, you can see this in the workspace's Cargo.toml file:

[profile.release]
lto = true
strip = true

[profile.dev]
strip = "debuginfo"
incremental = false

@ThibsG ThibsG marked this pull request as ready for review February 6, 2024 13:03
@ThibsG ThibsG requested a review from Manuthor February 6, 2024 13:04
@ThibsG ThibsG force-pushed the ci_optimizations branch 2 times, most recently from 2ff393c to 819a7e0 Compare February 7, 2024 06:56
@ThibsG ThibsG changed the title CI optimizations CI improvements Feb 7, 2024
@Manuthor Manuthor merged commit 762a15c into develop Feb 7, 2024
@Manuthor Manuthor deleted the ci_optimizations branch February 7, 2024 10:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants