Support custom options for LLVM build

[tmandry]

The LLVM build has a lot of options that rustbuild doesn't need to know about. We should allow the user to customize the LLVM build directly.

Here are some example customizations we'd like to do.

[rust-highfive]

r? @Mark-Simulacrum

(rust-highfive has picked a reviewer for you, use r? to override)

[Mark-Simulacrum]

Hm. So, part of me is enthusiastic about supporting something like this, on the other hand, I wonder if it's worth some kind of disclaimer of "at your own risk" or similar -- part of the strategy I've at least sort of tried to steer by is that if folks are trying to extra-customize LLVM, they may just want to build out of tree entirely and provide an llvm-config. (Obviously, that has its own problems).

But I think this is sufficiently general and minimally invasive that I am in general on board with it -- I do wonder if you might be able to ask around and/or have some ideas on other projects embedding LLVM to see if there's some general guidance around the appropriate way to expose this? In particular, with my relatively limited knowledge I'm wondering if just CMAKE defines are typically enough? Or do we need support for general environment configuration / something beyond that?

I will also add that if we do land this it's pretty likely we'll want to drop support for at least some of the existing flag in config.toml that we aren't ourselves interested in using/supporting (e.g., polly build).

[Mark-Simulacrum]

cc #93629 and #93640 as well, which seem like prime PRs I would "not accept" in favor of using this feature, though as noted on one of those I might have pushed for just leaving LLVM builds external as well.

@catap -- curious if you feel this PR would be enough for you or it's missing something you'd need?

[catap]

@Mark-Simulacrum definitely!

[Mark-Simulacrum]

r=me with the comment adjusted and llvm_from_ci! gate added.

[tmandry]

@bors r=Mark-Simulacrum

[bors]

📌 Commit 69cd826 has been approved by Mark-Simulacrum

[tmandry]

@bors rollup=always

[jonhoo]

I think this also needs an update to src/bootstrap/configure.py to allow setting this property through an argument, since as far as I can tell it doesn't have a meaningful way to set dictionaries at the moment.

[jonhoo]

Ah, or perhaps it'll just work by including the key for the dictionary in --set (so --set llvm.build-config.LLVM_ENABLE_ZLIB=FORCE_ON for example)?

[catap]

@jonhoo you may use ./configure --set='llvm.build-config={A = 2, C = D}' when you need multiple options

[Mark-Simulacrum]

I would expect --set llvm.config.foo=bar to work - do we have something hard coded to two elements in the path?

[catap]

@Mark-Simulacrum unfortently it is a bit more tricky. Special when someone needs a few LLVM options. Anyway, some ugly configure set make the job done.

[jonhoo]

Yeah, I just tried:

--set "llvm.build-config.LLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=1" \
--set "llvm.build-config.LLVM_ENABLE_ZLIB=FORCE_ON"

and got

Traceback (most recent call last):
  File "./src/bootstrap/configure.py", line 467, in <module>
    configure_section(sections[section_key], section_config)
  File "./src/bootstrap/configure.py", line 447, in configure_section
    lines[i] = "{} = {}".format(key, to_toml(value))
  File "./src/bootstrap/configure.py", line 436, in to_toml
    raise RuntimeError('no toml')
RuntimeError: no toml

[catap]

@jonhoo --set='llvm.build-config={LLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=1,LLVM_ENABLE_ZLIB=FORCE_ON}' works as you expected

[jonhoo]

I can also confirm that I get the same error even if I just set a single key this way. Will try your way now @catap.

[jonhoo]

@catap That doesn't look quite right either. It gives me

# Custom CMake defines to set when building LLVM.
build-config = '{LLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=1,LLVM_ENABLE_ZLIB=FORCE_ON}'

Which sets build-config to a string, not a dictionary, which will then fail to deserialize into a Map when the bootstrap logic picks it up.

[jonhoo]

I don't think that's true — I'm running it in bash, and I think that's the right quoting for the command. The problem is that set expects the value to always be set to either a string, a list, or bool. There's no support for emitting it as a dict.

[catap]

@jonhoo let me dig, how I've used it. You're right.

[jonhoo]

Separately, I also noticed that the way this is currently implemented, llvm.build-config doesn't apply to the LLD and sanitizer buildes. I wonder if

rust/src/bootstrap/native.rs

Lines 374 to 376 in 89adcc6

	for (key, val) in &builder.config.llvm_build_config {
	cfg.define(key, val);
	}

should move to

rust/src/bootstrap/native.rs

Line 415 in 89adcc6

fn configure_cmake(

which all of the three LLVM-based steps use?