rust-analyzer fails to load proc-macros when loaded through ld.so

[jonhoo]

This is a somewhat niche problem, but I figured there should at least be a public record of it.

If rust-analyzer is executed through ld.so, it fails to load the shared library for proc-macros:

proc-macro: error while loading shared libraries: proc-macro: cannot open shared object file

You can reproduce this with:

cargo new --lib ra-ldso
cd ra-ldso
/lib/"ld-linux-$(uname -m).so.1" "$(which rust-analyzer)" analysis-stats .

Note that it's not even necessary to actually pass --library-path — invoking through ld.so is sufficient.

Loading through ld.so is primarily useful because it allows setting the shared library search path without setting LD_LIBRARY_PATH. This, in turn, is useful because LD_LIBRARY_PATH is "viral" — it also affects the search path for child processes, whereas ld.so --library-path does not. It's important to use a non-viral mechanism in nix-style applications where each program lives in its own runtime environment, and you want a way to set the runtime environment for a given binary, but not for any binaries that it may in turn execute, since those may themselves have their won runtime environments.

Based purely on a guess, I suspect this is related to the solution implemented through #12803, but hard to say for sure. Alternatively, it might be related to how rust-analyzer locates its own binary to execute the proc-macro-srv — if it uses argv0, then it'll now find ld.so rather than the true rust-analyzer!

rust-analyzer version: rust-analyzer 0.0.0 (ab068f120 2022-08-31)

rustc version: rustc 1.63.0 (4b91a6ea7 2022-08-08)

[Veykril]

Can you check your server logs and paste them here? They should have some more info potentially (I don't have a linux machine at hand)

[jonhoo]

$ RA_LOG=lsp_server=debug /lib/ld-linux-aarch64.so.1 "$(which rust-analyzer)" analysis-stats .
Failed to create perf counter: Operation not permitted (os error 1)
Failed to create perf counter: Operation not permitted (os error 1)
proc-macro: error while loading shared libraries: proc-macro: cannot open shared object file
Database loaded:     481.40ms (metadata 383.73ms; build 55.34ms)
Failed to create perf counter: Operation not permitted (os error 1)
  crates: 1, mods: 2, decls: 3, fns: 2
Item Collection:     3.90s
Failed to create perf counter: Operation not permitted (os error 1)
  exprs: 35, ??ty: 0 (0%), ?ty: 0 (0%), !ty: 0
Inference:           1.97s
Total:               5.86s

Or are you looking for something more than this?

[Veykril]

Can you set the RA_LOG to warn,rust_analyzer::reload=debug,project_model::build_scripts=info? I think that should have the important things

[jonhoo]

$ env RA_LOG=warn,rust_analyzer::reload=debug,project_model::build_scripts=info /lib/ld-linux-aarch64.so.1 $(which rust-analyzer) analysis-stats .
Failed to create perf counter: Operation not permitted (os error 1)
Failed to create perf counter: Operation not permitted (os error 1)
[INFO project_model::build_scripts] Running build scripts: "cargo" "check" "--quiet" "--workspace" "--message-format=json" "--all-targets"
proc-macro: error while loading shared libraries: proc-macro: cannot open shared object file
Database loaded:     486.08ms (metadata 386.89ms; build 55.84ms)
Failed to create perf counter: Operation not permitted (os error 1)
  crates: 1, mods: 2, decls: 3, fns: 2
Item Collection:     3.96s
Failed to create perf counter: Operation not permitted (os error 1)
  exprs: 35, ??ty: 0 (0%), ?ty: 0 (0%), !ty: 0
Inference:           2.17s
Total:               6.13s

[jonhoo]

And in case it's useful:

$ cargo check --quiet --workspace --message-format=json
{"reason":"compiler-artifact","package_id":"ra-test 0.1.0 (path+file:///local/home/jongje/ra-test)","manifest_path":"/local/home/jongje/ra-test/Cargo.toml","target":{"kind":["lib"],"crate_types":["lib"],"name":"ra-test","src_path":"/local/home/jongje/ra-test/src/lib.rs","edition":"2021","doc":true,"doctest":true,"test":true},"profile":{"opt_level":"0","debuginfo":2,"debug_assertions":true,"overflow_checks":true,"test":false},"features":[],"filenames":["/local/home/jongje/ra-test/target/debug/deps/libra_test-6c441a19cebcf54a.rmeta"],"executable":null,"fresh":true}
{"reason":"build-finished","success":true}

[Veykril]

Oh right, analysis-stats is not running the server. Ye so I think your assumption might be right, analysis-stats is always running the "current exe" with proc-macro as the arg unlike the server that now uses the sysroot if possible

rust-analyzer/crates/rust-analyzer/src/cli/load_cargo.rs

Lines 60 to 66 in 643c3a5

	let proc_macro_client = if load_config.with_proc_macro {
	let path = AbsPathBuf::assert(std::env::current_exe()?);
	Ok(ProcMacroServer::spawn(path, &["proc-macro"]).unwrap())
	} else {
	Err("proc macro server not started".to_owned())
	};

Though #12803 is unrelated to that as this was already the behavior prior to that

[Veykril]

So when using r-a as the server this should work fine (as long as the rust sysroot is recent enough), otherwise this fails with the same problem as we fall back to the current exe.

There is a config (that analysis-stats doesn't use since it isn't configurable) to set the proc-macro server path, though that is a bit odd as well as it currently works relative to the workspace you have opened (we should fix that, but haven't yet).

[jonhoo]

Hmm, I'm seeing exactly the same failure when using rust-analyzer through ld.so just as an LSP server (with neovim LSP as the client)..

[bjorn3]

When rust-analyzer tries to load proc macros, it does so by dlopen'ing them in a newly spawned process. This process is not created by invoking the same ld.so as you did for the main rust-analyzer process. Also the libc version needs to match (or be newer than) what rustc compiled the proc macro against.

if it uses argv0, then it'll now find ld.so rather than the true rust-analyzer!

It does either that or /proc/self/exe (through std::env::current_exe()).

By the way why aren't you using patchelf?

[jonhoo]

When rust-analyzer tries to load proc macros, it does so by dlopen'ing them in a newly spawned process. This process is not created by invoking the same ld.so as you did for the main rust-analyzer process.

That doesn't explain the problem I don't think, since rust-analyzer fails when invoked this way even if no arguments are provided to ld.so. That is, I'm not actually overriding anything.

Also the libc version needs to match (or be newer than) what rustc compiled the proc macro against.

Yep, that should be the case here.

if it uses argv0, then it'll now find ld.so rather than the true rust-analyzer!

It does either that or /proc/self/exe (through std::env::current_exe()).

Actually, looking at it again, /proc/self/exe will point to ld.so in this situation. It looks like "fixing" argv0 to point to rust-analyzer doesn't fix the problem:

$ sh -c "exec -a $(which rust-analyzer) /lib/ld-linux-aarch64.so.1 $(which rust-analyzer) analysis-stats ."
...
proc-macro: error while loading shared libraries: proc-macro: cannot open shared object file

Which suggests that rust-analyzer is indeed relying just on /proc/self/exe (which leads it astray with ld.so).

By the way why aren't you using patchelf?

patchelf would set RPATH, which either requires an absolute path (which would make the binary impossible to run on other hosts) or requires using $ORIGIN, which resolves symlinks before relative paths, which in turn breaks things in my setup where many of the binaries (and shared libraries) will be symlinked into a common directory to construct the runtime environment.

[jonhoo]

For some related context, see also rust-lang/cargo#10115 and rust-lang/cargo#10113.

[bjorn3]

I believe the NixOS version of rustup automatically runs patchelf on all binaries it downloads. That should be fine, right? It doesn't matter that the path is absolute as you won't copy the toolchain to another machine anyway. There is a rustup component for rust-analyzer and even a rustup component for just the proc macro server of rust-analyzer (not sure if it is available on stable already. it is available on beta) in case you want to keep using non-rustup versions of rust-analyzer. It should automatically load the proc macro server from the toolchain if available.

[jonhoo]

Ah, so, to be clear, while the system I'm working within (Amazon's build system) shares some similarities with Nix, it is not Nix. Nix has the advantage that paths are the same across hosts, so setting absolute paths in RPATH is fine. We do not have that luxury sadly. And we do want to be able to use the same runtime environments across different hosts.

There's also no rustup involved here — the value from $(which rust-analyzer) is a path to a standalone rust-analyzer binary (from cargo xtask install).

[Veykril]

Out of curiosity, what rust version are you working with right now? (Should've been clearer on this) If you are on stable, then yes the rust-analyzer server itself will also fail as the sysroot change hasnt landed there yet, that is currently only on recent nightlies

[jonhoo]

Yes, this is with stable Rust, but with rust-analyzer build directly from the latest git release tag.