Figure out pstack and Rust strip = "symbols" interaction

[sunshowers]

(writing up a quick summary so I don't forget about this)

@jclulow reported to me that a hung nextest process didn't show its symbols in pstack. It turned out that was because the build process we were using within nextest was stripping all symbols.

I tried switching to strip = "debuginfo" but that didn't seem to help. The only thing that caused stacks to show up in pstack was strip = "none".

In nextest-rs/nextest@d4f982b#commitcomment-140289873, Taiki believes that this is a bug (there was apparently a similar bug in MSVC as well.) I tried setting up a small project to investigate:

https://github.com/sunshowers/pstack-test

but found that even with the strip-symbols profile (which activates strip = "symbols"), pstack could show function symbols. Not quite clear why that would be happening though!

[taiki-e]

but found that even with the strip-symbols profile (which activates strip = "symbols"), pstack could show function symbols. Not quite clear why that would be happening though!

Does this mean there is a problem only with strip=debuginfo?

Also, is this a problem that only occurs with cross-compilation?

If both are true, since only strip=debuginfo use /usr/bin/strip, that would be consistent with my guess in nextest-rs/nextest@d4f982b#commitcomment-140325483 that non-illumos host's /usr/bin/strip are causing some problems.

[sunshowers]

Does this mean there is a problem only with strip=debuginfo?

Ah no, I mean that no matter what I set strip to, I couldn't reproduce the results.

I do think this is likely a cross-compilation issue as you pointed out -- I was doing my builds natively on illumos but the CI that runs strip was doing a cross-compile from Linux.

[sunshowers]

@taiki-e -- so what can we do here? Is this something where rustc can help, or maybe cross?

[taiki-e]

For now, I think it makes sense to wait for discussion at the next compiler-team meeting (rust-lang/rust#123151 (comment)).

If the decision is made to use llvm-strip, the fix will likely be backported to stable (as it also affects macOS, which is tier 1 target).

[workingjubilee]

Huh, reading this: https://illumos.org/books/dev/debugging.html

Let's take a look now at pstack. As long as sufficient information for debugging is present in a binary, pstack can tell you what's going on. pstack doesn't rely on DWARF, it simply needs to have access to the symbol table and a frame pointer. If it's software in illumos, you can rest assured that it comes this way by default.

pstack doesn't rely on DWARF? oh dear... I think our backtrace testing infra assumes that you're using the DWARF-centric unwinding and symbolication strategy if you're a Unix...