Skip to content

[SR-8857] llvm build tries to link too many things at a time #51363

Open
@kevints

Description

@kevints
Previous ID SR-8857
Radar None
Original Reporter @kevints
Type Bug
Additional Detail from JIRA
Votes 0
Component/s Compiler
Labels Bug, CMake
Assignee None
Priority Medium

md5: d5434fe783df9cc3bbb7a6a3908fcb3c

Issue Description:

Running the default build-script invocation from a clean workspace on Ubuntu 14.04 frequently results in a linker command crashing due to no memory. For example:

[2548/2838] Linking CXX executable bin/clang-import-test
FAILED: : && /usr/bin/clang++-3.5  -Wno-unknown-warning-option -Werror=unguarded-availability-new -g -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -fcolor-diagnostics -fno-common -Woverloaded-virtual -Wno-nested-anon-types -g  -Wl,-allow-shlib-undefined    -Wl,-rpath-link,/code/build/Ninja-DebugAssert/llvm-linux-x86_64/./lib tools/clang/tools/clang-import-test/CMakeFiles/clang-import-test.dir/clang-import-test.cpp.o  -o bin/clang-import-test  -Wl,-rpath,"\$ORIGIN/../lib" lib/libLLVMCore.a lib/libLLVMSupport.a -lpthread lib/libclangAST.a lib/libclangBasic.a lib/libclangCodeGen.a lib/libclangDriver.a lib/libclangFrontend.a lib/libclangLex.a lib/libclangParse.a lib/libclangDriver.a lib/libLLVMOption.a lib/libclangSerialization.a lib/libclangSema.a lib/libclangAPINotes.a lib/libclangEdit.a lib/libclangAnalysis.a lib/libclangAST.a lib/libclangLex.a lib/libclangBasic.a lib/libLLVMCoroutines.a lib/libLLVMCoverage.a lib/libLLVMLTO.a lib/libLLVMObjCARCOpts.a lib/libLLVMPasses.a lib/libLLVMipo.a lib/libLLVMIRReader.a lib/libLLVMAsmParser.a lib/libLLVMLinker.a lib/libLLVMInstrumentation.a lib/libLLVMVectorize.a lib/libLLVMCodeGen.a lib/libLLVMBitWriter.a lib/libLLVMScalarOpts.a lib/libLLVMInstCombine.a lib/libLLVMTarget.a lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a lib/libLLVMObject.a lib/libLLVMBitReader.a lib/libLLVMMCParser.a lib/libLLVMMC.a lib/libLLVMProfileData.a lib/libLLVMCore.a lib/libLLVMBinaryFormat.a lib/libLLVMSupport.a -lz -lrt -ldl -ltinfo -lpthread -lm lib/libLLVMDemangle.a && :
clang: error: unable to execute command: Killed
clang: error: linker command failed due to signal (use -v to see invocation)

I am running build-script with no arguments, so I would expect it to just work on the first run. The system is a Docker for Mac VM with 8 cores and 16GiB of RAM assigned to it (the maximum the slider will go to). Build script detects that I have 8 cores and issues -j8. If I'm unlucky it will try to link 8 different LLVM libraries at the same time, and one will crash with out-of-memory, failing the build. One way to improve this would be to use CMake's JOB_POOL_LINK [1] feature to direct link jobs into a job pool with a smaller number of workers available, say 1 or 2.

[1]: https://cmake.org/cmake/help/v3.0/prop_tgt/JOB_POOL_LINK.html#prop_tgt:JOB_POOL_LINK

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugA deviation from expected or documented behavior. Also: expected but undesirable behavior.cmakecompilerThe Swift compiler itself

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions