[vcpkg-tool-gn] Build gn from source on 16 KiB page arm64-Linux

[ADKaster]

This allows users with Asahi Linux machines to build vcpkg ports that require the gn build tool.

Fixes ##39468

• Changes comply with the maintainer guide.
• SHA512s are updated for each updated download.
• The "supports" clause reflects platforms that may be fixed by this new version.
• Any fixed CI baseline entries are removed from that file.
• Any patches that are no longer applied are deleted from the port's directory.
• The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
• Only one version is added to each modified port's versions file.

This approach... works, but might not be the desired direction? This appears to be the first "host tool" that actually wants to build native code. The way to get a compiler for gn to use is a bit sketchy.

[ADKaster]

@microsoft-github-policy-service agree

[dg0yt]

(community feedback)

• I would probably put building from source into a separate cmake script.
• I would not check for pagesize at all. Doing that in script mode is clumsy and duplicates build-related code. A proper implementation would call the vcpkg function to build a test project.
  Simply select build from soure when no matching binary is available.

[ADKaster]

Simply select build from source when no matching binary is available.

I'm not sure how this approach would fix the linked issue, #39468

The problem there is that while there is a linux-arm64 binary available, it is unusable if the system page size is set to 16 KiB (a kernel config value). So we need a way to detect that the existing linux-arm64 binary will not be usable on this platform, and to build from source instead.

Is there another way to detect this property of the host system at script time that would be better?

I do know that the information is available in the /proc/ filesystem on linux. So if we only want it to be checked for Linux arm64, we could look at /proc in a way like this:

$ grep -i pagesize /proc/self/smaps | sort | uniq
KernelPageSize:        4 kB
MMUPageSize:           4 kB

or via getconf(1)

$ getconf PAGESIZE
4096

[dg0yt]

Sorry, I didn't pay enought attention. IMO there are three alternatives:

• a feature (needs discussion; covers other platforms)
• a triplet variable (easy; covers other platforms)
• a small build with maintainer functions (still quite easy; but limited platform coverage).

The second option only needs

if(VCPKG_TOOL_GN_FROM_SOURCE) # or similar
   include(build.cmake)
else()
   # do as before
endif()

in the portfile, and

set(VCPKG_TOOL_GN_FROM_SOURCE 1)

in the custom triplet. It could be easily transformed to the first option.

Community feedback. Looking forward to hear from the MS team.

[Neumann-A]

This just runs into the tool ports blocker.

[BillyONeal]

@AugP
@ras0219-msft
@JavierMatosD
and I

discussed this today.

1. What makes gn special and lets the other aarch64 things we expect to download work? We aren't saying that we necessarily need a complete solution to everything about this before this can work but we need to understand the problem space to evaluate the correctness of this change. Do we need to look at configuration information from the compiler itself? (For example, extending detect_compiler to tell us more things)

In particular we need to have high confidence that it isn't going to break binary caching. Ideally we also know the end to end story works before we make incremental steps.

3. Sorry, I didn't pay enought attention. IMO there are three alternatives:

• a feature (needs discussion; covers other platforms)
• a triplet variable (easy; covers other platforms)
• a small build with maintainer functions (still quite easy; but limited platform coverage).

It seems like this needs to be an entirely different 'cpu' because this will affect vcpkg fetch and friends too.

[ADKaster]

What makes gn special and lets the other aarch64 things we expect to download work?

@BillyONeal

My understanding of the reason that gn has issues on 16 KiB Linux kernels and other tools do not is a combination of factors:

• the gn binaries that the chromium package sources ship are fully statically linked
• gn uses the jemalloc allocator. Or at least, the binaries shipped by the chromium package repos do. Jemalloc uses the system page size as a compile time constant. If the system page size and the page size that jemalloc was compiled for do not match at runtime, it is a fatal error.

The other tools such as CMake, ninja, meson, etc that have aarch64 binaries downloaded from a third party do not statically link jemalloc, or another allocator that hard codes the system page size at compile time.

[ADKaster]

Another alternative could be to create infrastructure for vcpkg to build from source and cache tool binaries on GitHub or another CDN that the upstream doesn't ship.

[dg0yt]

You don't need it.

• Make a project subdir with regular CMakeLists.txt.
• Use vcpkg_cmake_configure(SOURCE_PATH "${CURRENT_PORT_DIR}/project" ...) - it will pass the actual toolchain settings.
  Make the configuration always succeed, and make it write a result file (aligned via OPTIONS "-DRESULT_FILE=${CURRENT_BUILDTREES_DIR}/result-${TARGET_TRIPLET}.cmake") with the desired information.
• Load the result file.

[LilyWangLL]

Ping @ADKaster for response. Can you address the review suggestions? Thanks!

[ADKaster]

Hi @LilyWangLL , sorry for the late reply. I don't think I'll have the capacity to address the comments on this issue.