Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Python CI: More swap space #1250

Merged
merged 1 commit into from
Jul 21, 2022
Merged

Conversation

gchenfc
Copy link
Member

@gchenfc gchenfc commented Jul 20, 2022

Recent CI's have been failing due to Python compile needing large amounts of RAM. Specifically, gcc in DEBUG mode appears to be pushing the CI over the edge. e.g.

[ 93%] Built target gtsam_unstable_py
[ 93%] Building CXX object python/CMakeFiles/gtsam_py.dir/slam.cpp.o
g++-5: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See file:///usr/share/doc/gcc-5/README.Bugs for instructions.
make[2]: *** [python/CMakeFiles/gtsam_py.dir/nonlinear.cpp.o] Error 4
python/CMakeFiles/gtsam_py.dir/build.make:257: recipe for target 'python/CMakeFiles/gtsam_py.dir/nonlinear.cpp.o' failed
make[2]: *** Waiting for unfinished jobs....
CMakeFiles/Makefile2:6887: recipe for target 'python/CMakeFiles/gtsam_py.dir/all' failed
make[1]: *** [python/CMakeFiles/gtsam_py.dir/all] Error 2
make: *** [all] Error 2
Makefile:165: recipe for target 'all' failed
Error: Process completed with exit code 2.

Varun/Fan/I discussed this issue.

This PR increases the (Linux) swap space from (default) 4GB to 6GB. Alternatively, we could also reduce number of threads from -j2 to -j1 but that will ~double build times.

Also for reference, currently hard drive usage is very low, only 4.1G / 14G. Here's output of df after python DEBUG build on python CI:

runner@fv-az199-378:~/work/gtsam/gtsam$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            3.4G     0  3.4G   0% /dev
tmpfs           696M  740K  695M   1% /run
/dev/sdb1        84G   59G   25G  71% /
tmpfs           3.4G     0  3.4G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           3.4G     0  3.4G   0% /sys/fs/cgroup
/dev/sdb15      105M  4.4M  100M   5% /boot/efi
/dev/sda1        14G  4.1G  9.0G  31% /mnt
tmpfs           696M     0  696M   0% /run/user/1001

Copy link
Collaborator

@varunagrawal varunagrawal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweeeet

@varunagrawal varunagrawal merged commit e5c47b3 into develop Jul 21, 2022
@varunagrawal varunagrawal deleted the fix/ci-python-swapspace branch July 21, 2022 00:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants