tc-build supports building from tarballs

[dileks]

Does tc-build supports building from tarballs?

If yes , where do I have to place/unpack the tarballs speaking of the directory structure?

Currently, I see version 10.0.0-rc6 and wanted to do a stage1 build with LLVM, Clang and LLD tarballs.

Thanks to Universe the Clang tarball has no more cfe as prefix.

[1] https://prereleases.llvm.org/10.0.0/
[2] https://prereleases.llvm.org/10.0.0/rc6/llvm-10.0.0rc6.src.tar.xz
[3] https://prereleases.llvm.org/10.0.0/rc6/clang-10.0.0rc6.src.tar.xz
[4] https://prereleases.llvm.org/10.0.0/rc6/lld-10.0.0rc6.src.tar.xz

[nathanchance]

No, it does not. Just pass in --branch llvmorg-10.0.0-rc6 to check out the git repository that those tarballs were generated from.

[dileks]

Can you tell me how big such a git-checkout is?
Currently, I am using UMTS/LTE to connect to the Internet, so transferred data volume matters for me.

[nathanchance]

A full checkout is over a GB but a shallow clone (which can be done via --shallow-clone) is 123.25 MiB.

Alternatively, this works for me (assuming llvm-project does not already exist):

$ mkdir -p llvm-project

$ cd llvm-project

$ for PROJECT in llvm clang lld; do mkdir -p "${PROJECT}" && curl -LSs https://github.com/llvm/llvm-project/releases/download/llvmorg-10.0.0/"${PROJECT}"-10.0.0.src.tar.xz | tar -C "${PROJECT}" --strip-components=1 -xJf -; done

$ cd ..

$ ./build-llvm.py --build-stage1-only --no-update --projects "clang;lld"

[dileks]

Cool, LLVM version 10.0.0 was released 5 hours ago :-).

Did not know of --shallow-clone git-clone option, Thanks.

Will try later - both solutions.

[1] https://github.com/llvm/llvm-project/releases/tag/llvmorg-10.0.0

[dileks]

Sizes, please:

34.1 MB llvm-10.0.0.src.tar.xz
13.5 MB clang-10.0.0.src.tar.xz
1.09 MB lld-10.0.0.src.tar.xz

You happen to know what is included here?

80.5 MB llvm-project-10.0.0.tar.xz

[nathanchance]

llvm-project is https://github.com/llvm/llvm-project/tree/llvmorg-10.0.0 compressed.

[dileks]

So, it is simpler and "cleaner" to download llvm-project-10.0.0.tar.xz which includes all projects means Clang and LLD what I want.

[nathanchance]

Yes.

mkdir -p llvm-project && curl -LSs https://github.com/llvm/llvm-project/releases/download/llvmorg-10.0.0/llvm-project-10.0.0.tar.xz | tar -C llvm-project --strip-components=1 -xJf -

then just simply adding --no-update to your regular build-llvm.py invocation.

[dileks]

Cool.

The new llvm-project Git infrastructure offers a lot of new opportunities.

I was thinking of compiler regression testing with final releases.

What about offering a "tarball download" option (with setting "--no-update")?

[1] https://github.com/llvm/llvm-project/releases/download/llvmorg-9.0.1/llvm-project-9.0.1.tar.xz
[2] https://github.com/llvm/llvm-project/releases/download/llvmorg-10.0.0/llvm-project-10.0.0.tar.xz

[nathanchance]

I'd rather not complicate the script with another download option. --shallow-clone is available for most resource conscious scenarios (be it time or bandwidth). If that does not work, I feel that manually managing llvm-project and passing in --no-update is a completely reasonable workflow. I don't want the script to do everything from a source management perspective. I might be able to document this somewhere (README, I guess?).

[dileks]

Yeah, you are right - KISS.

Can you comment on the --shallow-clone option - how to do that?
My git version 2.25.1 does not know of git clone --shallow-clone.

[nathanchance]

Add --shallow-clone to ./build-llvm.py for it to take care of it; otherwise, clone llvm-project with --depth=1.

[dileks]

OK, I see this is a feature of tc-build not git.

Please, document this in README.md.

[nathanchance]

There is a -h/--help option for clear documentation of all of these flags, which is mentioned in the README.