recognizing ENZYME_RUNPASS for lib/LLVM Enzyme builds

[ZuseZ4]

I noticed that the ENZYME_RUNPASS variable is ignored, when you just pass it as -DENZYME_RUNPASS to cmake.
This might also affect your BUILD file, idk.
With these changes it now does run the augmentedPassBuilder.

[wsmoses]

This should be disabled for llvmenzyme thougj? The flag says that the enzyme pass should be injected to be run by default in the pass pipeline

[ZuseZ4]

What is the difference between libEnzyme-X.so and LLVMEnzyme-19.so? I thought they would have just different linking behaviour, but the same functionality? We use libEnzyme because it fixed some linking issues I can't remember, so I don't mind only updating it and removing the LLVMEnzyme one (?)

[wsmoses]

LLVMEnzyme is an llvm pass plugin, libenzyme is a dynamic library you can link into programs that aren't llvm's opt

[ZuseZ4]

Fair, I vaguely had in mind that at some point I also linked against LLVMEnzyme when using the Enzyme API, but I might have confused something.

Also, remember you'll need to merge, and can't just approve things.

[wsmoses]

I'm still not understanding why this is needed here

[ZuseZ4]

I want Enzymes opt pass to run befire autodiff, as it is supposed to improve performance. Right now Rust already sets -DENZYME_RUNPASS when calling cmake, but the value is ignored and does not cause Enzyme to run the optimization pass. It only affects libEnzyme, as lld and clangenzyme have it hardcoded. I could also hardcode it for libEnzyme, but this solution allows other users to run the autodiff pass without the opt pass.

[ZuseZ4]

Enzyme/enzyme/Enzyme/Enzyme.cpp

Line 3662 in 3ea5fe2

#ifdef ENZYME_RUNPASS

[wsmoses]

Why not just run augmentPassBuilder?

[ZuseZ4]

Why have RustEnxyme differ from ClangEnzyme or LLDEnzyem?

[wsmoses]

The use you linked is specific to llvm pass loading which doesn't make sense for API users, since its better form to build the pass manager directly (like with the function I mentioned)

[ZuseZ4]

This has been discussed e.g. here. There I had asked which functions I should use in which order. I ended up staring with another llvm dev at the enzyme code and this is what we came up with.

From what we saw, we will always run augmentPassBuilder and then registerEnzyme to then run the enzyme pass.
We will never just run augmentPassBuilder or the enzyme pass without the other. So if there is a performance benefit to run augmentPassBuilder separately, it would be good if you could give an explanation on how it's supposed to be used and what is the difference over just enabling it here. Because based on your explanation I would just run

augmentPassBuilder(PB);
registerEnzyme(PB); // With the augmentPassBuilder configured out
    if (auto Err = PB.parsePassPipeline(MPM, "enzyme")) {
      std::string ErrMsg = toString(std::move(Err));
      LLVMRustSetLastError(ErrMsg.c_str());
      return LLVMRustResult::Failure;
    }

I don't see how exposing and ffi-wrapping agumentPassBuilder is an improvement over the code I merged after the zulip discussions with other devs which assumes the augmentPassBuilder to implicitly run (which I try to fix here since it doesn't, because cmake dropps the ENZYME_RUNPASS flag for non-Clang/LLD builds):
https://github.com/rust-lang/rust/pull/136419/files#diff-eac3d5ef63f39b7914ae9015f31b8d87353ea8fbcd5b02a7335dd86c557e7625

[ZuseZ4]

Ok, wild guess now, but in an attempt to understand the difference, could it be that you missed the messages/pings where I/Rust people asked about the Enzyme pass?
Rust moved over to a pass-based Enzyme approach around October last year, based on your recommendation in some DMs, hence all my question about the enzyme pass since.
If you missed all of those this discussion suddenly makes a lot more sense to me and hopefully now to you.

[wsmoses]

I'm still a bit confused how you're intending to use this. In any case, perhaps you can make a second function void augmentAndRegisterEnzyme(PB, bool shouldAugment) which contains the current definition, and then is true if the current Enzyme_runpass is set. That way you can just call the api function without setting a compile time var

[ZuseZ4]

The intention is to have Rust differ as little as possible from Clang/LLD-Enzyme to ease maintenance, since all three are pass based, as opposed to Julia which is Api based., Clang/LLD Enzyme both define this variable to first run augmentPassBuilder and then register Enzyme and run it. So my confusion comes from your recommendation to handle Rust and C++ different here.
Doing it at runtime would be totally fine for me, but then I wonder why you don't do the same with the clang/lld plugin? What are you confused about?

[ZuseZ4]

Also for the long term goal, the idea is to have autodiff run as the pass (e.g. because of your recommendation and because it handles higher order derivatives in the right order automatically). For things like custom-derivatives you recommended to not copy c++Enzyme, but rather link against libEnzyme, so we can use the more advanced registerCustomDerivative functions, since they are more granular than the metadata approach.

So my understanding is that clang will in the future move to the api for custom derivatices, Enzyme.jl will move to the pass for autodiff (but keep using the api for things like custom derivatives) and then all 3 frontends will behave identical. Since Rust is coming in later, I directly try to do the right approach for each feature. Does that help?

[ZuseZ4]

@wsmoses adding the runtime variant and updated the old one to forward, to make sure they can never get out of sync.