WIP: PoC for linux built with LTO to reduce its size

[tlaurion]

This is a followup attempt to #590

Any help welcome

• Linux module is based on Andy's linux 4.20 LTO branch
  • Linux module requires lto flag to be passed
• Linux board's config needs to have LTO_MENU activated
• musl-cross needs to be compiled with lto support and plugins support

Still no joy.

Scattered documentation followed, from most informative to less

• Gentoo graphite + lto blog post
• Shrinking the kernel with an Axe blog post (3 article series)
• Followup on make-musl-cross commit deactivating lto support

[tlaurion]

Side notes which went nowhere, for traceability: richfelker/musl-cross-make@d2f59c9

[macpijan]

@tlaurion

What is the main problem here? Getting the toolchain to support the LTO? I have spinned up the build on your branch and it fails at:

  VDSO    arch/x86/entry/vdso/vdso64.so.dbg
/storage/projects/heads/heads/crossgcc/bin/x86_64-linux-musl-ld: arch/x86/entry/vdso/vclock_gettime.o: plugin needed to handle lto object
/storage/projects/heads/heads/crossgcc/bin/x86_64-linux-musl-ld: arch/x86/entry/vdso/vgetcpu.o: plugin needed to handle lto object
/storage/projects/heads/heads/crossgcc/bin/../lib/gcc/x86_64-linux-musl/8.3.0/../../../../x86_64-linux-musl/bin/nm: arch/x86/entry/vdso/vdso64.so.dbg: plugin needed to handle lto object

At first I have tried to simply build (outside of the heads build system) the kernel with/without LTO to see the potential gain.

For the GCC LTO I have used: https://github.com/andikleen/linux-misc/tree/lto-420-1
The build was on gcc 8.4.0 using the linux-x230 defconfig.

The results show some (~7%) gain:

-rw-r--r-- 1 maciej maciej 2929200 wrz 29 14:36 bzImage-420-gcc-with-lto
-rw-r--r-- 1 maciej maciej 3150384 wrz 29 14:22 bzImage-420-gcc-without-lto

I have also noticed quite recent approach with CLANG LTO: https://www.phoronix.com/scan.php?page=news_item&px=Linux-Kernel-Clang-LTO-Patches

In this case I have used kernel from https://github.com/samitolvanen/linux/tree/clang-lto
The toolchain was LLVM 11.
But in this case, size of the kernel built with LTO was greater than without (?):

-rw-rw-r-- 1 maciej maciej 2932000 wrz 29 14:38 bzImage-5.9rc7-clang-with-lto
-rw-rw-r-- 1 maciej maciej 2878480 wrz 29 13:39 bzImage-5.9rc7-clang-without-lto

[tlaurion]

Only early experiments on this PR,I was not able to get the toolchain to support LTO.

If the final binary is not getting cleaned post link phase, this means LTO is not enforced properly, as referred per referred articles.

If we get kernel size downsized, we might want to bring that to other large libraries and binaries of heads, where I have no idea how we should call the cleanup so it is only applied if no other modules uses it.