Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI Speedup #839

Merged
merged 8 commits into from
Aug 11, 2021
Merged

CI Speedup #839

merged 8 commits into from
Aug 11, 2021

Conversation

varunagrawal
Copy link
Collaborator

@varunagrawal varunagrawal commented Aug 6, 2021

Specialize Boost install so that the CI is faster.

This gives a ~17 minute improvement since Boost install goes down from ~22 mins to ~5 mins.

Shoutout to @ProfFan for making me aware of this!

@varunagrawal varunagrawal added the performance Related to GTSAM performance and efficiency label Aug 6, 2021
@varunagrawal varunagrawal self-assigned this Aug 6, 2021
@jlblancoc jlblancoc self-requested a review August 8, 2021 11:38
@varunagrawal
Copy link
Collaborator Author

Added some further performance improvements. This gives a 4x improvement on Linux, and 2x on macOS. Windows is still problematic (Release version takes 2 hours both before and after!), but we removed the dependency on the MarkusJx Boost action which is nice.
Some CI workflows have increased in time, but the increase is very tolerable (e.g. 18m to 21m) especially considering the gains we've made.

CI Job Previous Runtime New Runtime
Linux CI / ubuntu-18.04-gcc-5 Debug (push) 63m 14m
Linux CI / ubuntu-18.04-gcc-5 Release (push) 50m 13m
Linux CI / ubuntu-18.04-gcc-9 Debug (push) 69m 15m
Linux CI / ubuntu-18.04-gcc-9 Release (push) 62m 15m
Linux CI / ubuntu-18.04-clang-9 Debug (push) 50m 11m
Linux CI / ubuntu-18.04-clang-9 Release (push) 61m 14m
Special Cases CI / ubuntu-gcc-tbb Debug 62m 17m
Special Cases CI / ubuntu-gcc-tbb Release 64m 15m
Special Cases CI / ubuntu-cayleymap Debug 60m 16m
Special Cases CI / ubuntu-cayleymap Release 60m 14m
Special Cases CI / ubuntu-gcc-deprecated Debug 58m 14m
Special Cases CI / ubuntu-gcc-deprecated Release 61m 15m
Special Cases CI / ubuntu-gcc-quaternions Debug 66m 16m
Special Cases CI / ubuntu-gcc-quaternions Release 48m 16m
Python CI / ubuntu-18.04-gcc-5 Debug Python 3 22m 21m
Python CI / ubuntu-18.04-gcc-5 Release Python 3 16m 21m
Python CI / ubuntu-18.04-gcc-9 Debug Python 3 21m 22m
Python CI / ubuntu-18.04-gcc-9 Release Python 3 19m 21m
Python CI / ubuntu-18.04-clang-9 Debug Python 3 14m 11m
Python CI / ubuntu-18.04-clang-9 Release Python 3 17m 18m
Python CI / macOS-10.15-xcode-11.3.1 Debug Python 3 14m 19m
Python CI / macOS-10.15-xcode-11.3.1 Release Python 3 20m 23m
Python CI / ubuntu-18.04-gcc-5-tbb Debug Python 3 22m 18m
Python CI / ubuntu-18.04-gcc-5-tbb Release Python 3 20m 15m
Windows CI / windows-2019-cl Debug 14m 20m
Windows CI / windows-2019-cl Release 165m 176m
Linux CI / ubuntu-18.04-gcc-5 Debug 65m 13m
Linux CI / ubuntu-18.04-gcc-5 Release 56m 15m
Linux CI / ubuntu-18.04-gcc-9 Debug 64m 18m
Linux CI / ubuntu-18.04-gcc-9 Release 54m 11m
Linux CI / ubuntu-18.04-clang-9 Debug 45m 13m
Linux CI / ubuntu-18.04-clang-9 Release 49m 11m
macOS CI / macOS-10.15-xcode-11.3.1 Debug 30m 14m
macOS CI / macOS-10.15-xcode-11.3.1 Release 34m 14m

@varunagrawal varunagrawal merged commit 41c8268 into develop Aug 11, 2021
@varunagrawal varunagrawal deleted the feature/ci-speedup branch August 11, 2021 00:01
@dellaert
Copy link
Member

This is awesome! What was the biggest timesaver on Linux?

@varunagrawal
Copy link
Collaborator Author

For Linux, not installing all of Boost and using make -j$(nproc) gave the biggest savings. For Mac, I did the same trick for make but with the appropriate command to figure out number of cores on the system.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Related to GTSAM performance and efficiency
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants