[llvm-symbolizer] new pm regression, symbolization fails to recover line number with -ffunction-sections -gdwarf-aranges

[goussepi]

Bugzilla Link	52187
Version	unspecified
OS	Windows NT
Attachments	use-after-scope-capture.cpp
CC	@aeubanks,@dwblaikie,@jmorse,@pogo59,@pogo59

Extended Description

Asan's test case use-after-scope-capture.cpp fails if -ffunction-sections -gdwarf-aranges and new pm is enabled.

The test is failing as the stack trace does not contain line number anymore.

---

#include

int main() {
std::function<int()> f;
{
int x = 0;
f = &x attribute((noinline)) {
return x; // BOOM
// CHECK: ERROR: AddressSanitizer: stack-use-after-scope
// CHECK: #​0 0x{{.}} in {{.}}use-after-scope-capture.cpp:[[@LINE-2]]
};
}
return f(); // BOOM
}
-----------<

llvm-symbolizer-output:

---

$ clang++ -ffunction-sections -gdwarf-aranges -gline-tables-only -O1 use-after-scope-capture.cpp && llvm-nm a.out --print-size -C | grep operator | cut -d" " -f1 | sed 's/^/0x/' | xargs llvm-symbolizer -e a.out

main::$_0::operator()() const
use-after-scope-capture.cpp:0:0
-------------------<

[dwblaikie]

Yep, looks to be a bug in aranges+gmlt I'm guessing. But certtainly a but in aranges in some form, since the aranges don't match the CU's ranges:

          DW_AT_ranges [DW_FORM_sec_offset] (0x00000000
             [0x0000000000401110, 0x0000000000401148)
             [0x0000000000401150, 0x0000000000401158)
             [0x0000000000401160, 0x0000000000401187)
             [0x0000000000401190, 0x0000000000401196))

.debug_aranges contents:
Address Range Header: length = 0x0000004c, format = DWARF32, version = 0x0002, cu_offset = 0x00000000, addr_size = 0x08, seg_size = 0x00
[0x0000000000401110, 0x0000000000401148)
[0x0000000000401150, 0x0000000000401158)
[0x0000000000401160, 0x0000000000401187)

Side note: I'm curious: why are you using aranges? In my experience they add to debug info size, while providing little in the way of symbolizing (or other) performance compared to reading the ranges off the CUs in debug_info.

[dwblaikie]

Yeah, this probably came from my optimizations to gmlt a few years ago - omitting subprograms for entities that would be adequately described by the symbol table + line table, without the need for any DIE description.

LLVM's arange output is powered by the emission of the DIE attributes for the address, whereas the CU ranges are kept even if no DIE for the subrange is emitted.

(we should /probably/ just move to using the CU ranges to generate aranges, but there's some open questions about whether global variables should have their addresses in aranges - I don't think GCC does for instance. If they don't go in there, then aranges shoudl be identical to CU ranges - maybe in DWARFv6 we can spec a better aranges that shares the CU ranges... though I still wonder whether it's worthwhile compared to checking a CU's ranges directly)

[goussepi]

Thank you @​dblaikie for the analysis, I am not familiar with the details unfortunately, but we have tools making use of .debug_aranges so that's why we have -gdwarf-aranges on by default.

[dwblaikie]

but we have tools making use of .debug_aranges

That's the bit I'm curious about, if you can expand on it: what tools/why are they using/dependent on aranges (what would happen if you stopped passing -gdwarf-aranges? Could the tools use the ranges on compile_units in the debug_info?)

[goussepi]

That's the bit I'm curious about, if you can expand on it: what tools/why
are they using/dependent on aranges (what would happen if you stopped
passing -gdwarf-aranges? Could the tools use the ranges on compile_units in
the debug_info?)

As far as I can tell we use .debug_aranges purely as an acceleration structure.
The performance difference is apparently significant in the case of crash reporting systems that need to parse potentially many coredumps.

[dwblaikie]

That's the bit I'm curious about, if you can expand on it: what tools/why
are they using/dependent on aranges (what would happen if you stopped
passing -gdwarf-aranges? Could the tools use the ranges on compile_units in
the debug_info?)

As far as I can tell we use .debug_aranges purely as an acceleration
structure.
The performance difference is apparently significant in the case of crash
reporting systems that need to parse potentially many coredumps.

Yeah, that's what I'm curious about though - how does .debug_aranges improve performance compared to retrieving address ranges from CUs? (I really wouldn't've thought that difference in parsing a single CU DIE to find the same address range list would be significant - it makes me think there's something else at work and I'd like to understand more)

(but to be clear, none of this is critical to getting this bug fixed - still should be fixed so long as we're supporting aranges - but I continue to wonder why we support aranges as they don't seem like they offer much performance benefit to me, so if there's something I'm not understanding about their performance benefit I'd really like to understand it better)

[goussepi]

That's the bit I'm curious about, if you can expand on it: what tools/why
are they using/dependent on aranges (what would happen if you stopped
passing -gdwarf-aranges? Could the tools use the ranges on compile_units in
the debug_info?)

As far as I can tell we use .debug_aranges purely as an acceleration
structure.
The performance difference is apparently significant in the case of crash
reporting systems that need to parse potentially many coredumps.

Yeah, that's what I'm curious about though - how does .debug_aranges improve
performance compared to retrieving address ranges from CUs? (I really
wouldn't've thought that difference in parsing a single CU DIE to find the
same address range list would be significant - it makes me think there's
something else at work and I'd like to understand more)

(but to be clear, none of this is critical to getting this bug fixed - still
should be fixed so long as we're supporting aranges - but I continue to
wonder why we support aranges as they don't seem like they offer much
performance benefit to me, so if there's something I'm not understanding
about their performance benefit I'd really like to understand it better)

Sorry I do not have more details into the actual cause of the performances difference :-/
All I know is the performance regression was reported when we switched from a gcc 4.1 compatible toolchain to clang, as clang did not had .debug_aranges at the time.

[dwblaikie]

That's the bit I'm curious about, if you can expand on it: what tools/why
are they using/dependent on aranges (what would happen if you stopped
passing -gdwarf-aranges? Could the tools use the ranges on compile_units in
the debug_info?)

As far as I can tell we use .debug_aranges purely as an acceleration
structure.
The performance difference is apparently significant in the case of crash
reporting systems that need to parse potentially many coredumps.

Yeah, that's what I'm curious about though - how does .debug_aranges improve
performance compared to retrieving address ranges from CUs? (I really
wouldn't've thought that difference in parsing a single CU DIE to find the
same address range list would be significant - it makes me think there's
something else at work and I'd like to understand more)

(but to be clear, none of this is critical to getting this bug fixed - still
should be fixed so long as we're supporting aranges - but I continue to
wonder why we support aranges as they don't seem like they offer much
performance benefit to me, so if there's something I'm not understanding
about their performance benefit I'd really like to understand it better)

Sorry I do not have more details into the actual cause of the performances
difference :-/
All I know is the performance regression was reported when we switched from
a gcc 4.1 compatible toolchain to clang, as clang did not had .debug_aranges
at the time.

It'd be worth knowing more about - because one possible direction here would be to remove aranges support from LLVM/Clang (can't have buggy code if the code doesn't exist). If there are solid use cases for aranges, I'd love to know about them to appropriately prioritize functionality/design choices like this.

[pogo59]

Sorry I do not have more details into the actual cause of the performances
difference :-/
All I know is the performance regression was reported when we switched from
a gcc 4.1 compatible toolchain to clang, as clang did not had .debug_aranges
at the time.

It'd be worth knowing more about - because one possible direction here would
be to remove aranges support from LLVM/Clang (can't have buggy code if the
code doesn't exist). If there are solid use cases for aranges, I'd love to
know about them to appropriately prioritize functionality/design choices
like this.

So, we have a tool (part of what, from the name of it, would appear to be a
kind of automated crash analysis system) that depends on .debug_aranges to
tell it where to find the CU for a given address in the backtrace, so that
it can find source coordinates. They've agreed that it could be worth doing
the experiment to build an equivalent index from CU ranges, and see if the
performance meets their needs, which would let us reasonably drop generating
.debug_aranges in LLVM. But no promises when that might happen.

In the meantime... we're seeing a loss of functionality, and it would be nice
to get it back. Just a statement, not trying to argue that you personally
should be the one to do the work.

[dwblaikie]

Sorry I do not have more details into the actual cause of the performances
difference :-/
All I know is the performance regression was reported when we switched from
a gcc 4.1 compatible toolchain to clang, as clang did not had .debug_aranges
at the time.

It'd be worth knowing more about - because one possible direction here would
be to remove aranges support from LLVM/Clang (can't have buggy code if the
code doesn't exist). If there are solid use cases for aranges, I'd love to
know about them to appropriately prioritize functionality/design choices
like this.

So, we have a tool (part of what, from the name of it, would appear to be a
kind of automated crash analysis system) that depends on .debug_aranges to
tell it where to find the CU for a given address in the backtrace, so that
it can find source coordinates. They've agreed that it could be worth doing
the experiment to build an equivalent index from CU ranges, and see if the
performance meets their needs, which would let us reasonably drop generating
.debug_aranges in LLVM. But no promises when that might happen.

In the meantime... we're seeing a loss of functionality, and it would be nice
to get it back. Just a statement, not trying to argue that you personally
should be the one to do the work.

Yep yep. (does "we" include Pierre/are they the same user - or distinct?)

I feel some responsibility since I broke this with my improvements to -gmlt, but the arange support is certainly sketchy (doesn't correctly understand nodebug code, for instance - it records everything from one address emitted into some DIE to the next address emitted into some DIE as pertaining to the CU of the first DIE - without any concern for the real length of the entity, etc) so it's a bit of an awkward thing to "fix", as such.

[pogo59]

In the meantime... we're seeing a loss of functionality, and it would be nice
to get it back. Just a statement, not trying to argue that you personally
should be the one to do the work.

Yep yep. (does "we" include Pierre/are they the same user - or distinct?)

The "we" seeing a loss of functionality is Sony's overall ecosystem; it's
in a component that isn't the responsibility of the developer toolchain.
Pierre is the toolchain engineer who picked up the internal bug report.
I'm poking my nose in because it relates to debug info.

[goussepi]

In the meantime... we're seeing a loss of functionality, and it would be nice
to get it back. Just a statement, not trying to argue that you personally
should be the one to do the work.

Yep yep. (does "we" include Pierre/are they the same user - or distinct?)

The "we" seeing a loss of functionality is Sony's overall ecosystem; it's
in a component that isn't the responsibility of the developer toolchain.
Pierre is the toolchain engineer who picked up the internal bug report.
I'm poking my nose in because it relates to debug info.

Yes, just to confirm we are seeing a loss of functionality with Asan as we support symbolizing Asan stack traces with llvm-symbolizer.
I have noticed also that addr2line seems to be able to symbolize ok.

[dwblaikie]

In the meantime... we're seeing a loss of functionality, and it would be nice
to get it back. Just a statement, not trying to argue that you personally
should be the one to do the work.

Yep yep. (does "we" include Pierre/are they the same user - or distinct?)

The "we" seeing a loss of functionality is Sony's overall ecosystem; it's
in a component that isn't the responsibility of the developer toolchain.
Pierre is the toolchain engineer who picked up the internal bug report.
I'm poking my nose in because it relates to debug info.

Yes, just to confirm we are seeing a loss of functionality with Asan as we
support symbolizing Asan stack traces with llvm-symbolizer.
I have noticed also that addr2line seems to be able to symbolize ok.

Ah, thanks for all the context!

Not sure if addr2line uses aranges? Yeah, I just modified a file to have an empty aranges and addr2line still successfully used the debug info, so I guess it's not reading it at all. I wouldn't mind changing llvm-symbolizer to do that too...