Skip to content

[DWARFLinker] Handle empty sequences when processing DW_AT_LLVM_stmt_sequence attributes #132875

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 27, 2025

Conversation

alx32
Copy link
Contributor

@alx32 alx32 commented Mar 25, 2025

We previously assumed that every DW_AT_LLVM_stmt_sequence attribute has a corresponding sequence in the processed line table. However, this isn't always true. Some sequences can be removed by the linker if they are empty, as shown here. When an attribute refers to one of these removed sequences, there is no actual sequence for it to match. In such cases, we update the attribute to indicate that it is invalid and does not point to any sequence. This informs readers that the attribute should be ignored.

The newly modified test would have triggered the assert that is being removed in this patch.

@alx32 alx32 requested a review from clayborg March 25, 2025 04:23
@alx32 alx32 marked this pull request as ready for review March 25, 2025 04:23
@alx32 alx32 requested a review from JDevlieghere as a code owner March 25, 2025 04:23
@llvmbot
Copy link
Member

llvmbot commented Mar 25, 2025

@llvm/pr-subscribers-debuginfo

Author: None (alx32)

Changes

We previously assumed that every DW_AT_LLVM_stmt_sequence attribute has a corresponding sequence in the processed line table. However, this isn't always true. Some sequences can be removed by the linker if they are empty, as shown here. When an attribute refers to one of these removed sequences, there is no actual sequence for it to match. In such cases, we update the attribute to indicate that it is invalid and does not point to any sequence. This informs readers that the attribute should be ignored.

The newly modified test would have triggered the assert that is being removed in this patch.


Full diff: https://github.com/llvm/llvm-project/pull/132875.diff

4 Files Affected:

  • (modified) llvm/lib/DWARFLinker/Classic/DWARFLinker.cpp (+7-2)
  • (modified) llvm/test/tools/dsymutil/ARM/stmt-seq-macho.test (+9)
  • (modified) llvm/test/tools/dsymutil/Inputs/private/tmp/stmt_seq/stmt_seq_macho.exe ()
  • (modified) llvm/test/tools/dsymutil/Inputs/private/tmp/stmt_seq/stmt_seq_macho.o ()
diff --git a/llvm/lib/DWARFLinker/Classic/DWARFLinker.cpp b/llvm/lib/DWARFLinker/Classic/DWARFLinker.cpp
index f66773ad2e694..ae4cc6d85c120 100644
--- a/llvm/lib/DWARFLinker/Classic/DWARFLinker.cpp
+++ b/llvm/lib/DWARFLinker/Classic/DWARFLinker.cpp
@@ -2311,8 +2311,13 @@ void DWARFLinker::DIECloner::generateLineTableForUnit(CompileUnit &Unit) {
           uint64_t OrigStmtSeq = StmtSeq.get();
           // 1. Get the original row index from the stmt list offset.
           auto OrigRowIter = SeqOffToOrigRow.find(OrigStmtSeq);
-          assert(OrigRowIter != SeqOffToOrigRow.end() &&
-                 "Stmt list offset not found in sequence offsets map");
+          // Check whether we have an output sequence for the StmtSeq offset.
+          // Some sequences are discarded by the DWARFLinker if they are invalid
+          // (empty).
+          if (OrigRowIter == SeqOffToOrigRow.end()) {
+            StmtSeq.set(UINT64_MAX);
+            continue;
+          }
           size_t OrigRowIndex = OrigRowIter->second;
 
           // 2. Get the new row index from the original row index.
diff --git a/llvm/test/tools/dsymutil/ARM/stmt-seq-macho.test b/llvm/test/tools/dsymutil/ARM/stmt-seq-macho.test
index b5093ba767894..1dd1f61f1f7fb 100644
--- a/llvm/test/tools/dsymutil/ARM/stmt-seq-macho.test
+++ b/llvm/test/tools/dsymutil/ARM/stmt-seq-macho.test
@@ -41,12 +41,21 @@ ATTRIB int function2_copy2(int a) {
     int result = a - 22;
     return result;
 }
+
+struct logic_error {
+    logic_error(const char* s) {}
+};
+ 
+struct length_error : public logic_error {
+    __attribute__((noinline)) explicit length_error(const char* s) : logic_error(s) {}
+};
  
 int main() {
     int sum = 0;
     sum += function2_copy2(3);
     sum += function3_copy2(41);
     sum += function2_copy1(11);
+    length_error e("test");
     return sum;
 }
 EOF
diff --git a/llvm/test/tools/dsymutil/Inputs/private/tmp/stmt_seq/stmt_seq_macho.exe b/llvm/test/tools/dsymutil/Inputs/private/tmp/stmt_seq/stmt_seq_macho.exe
index 138c418aa37b2..4dd4ee8deb0b4 100755
Binary files a/llvm/test/tools/dsymutil/Inputs/private/tmp/stmt_seq/stmt_seq_macho.exe and b/llvm/test/tools/dsymutil/Inputs/private/tmp/stmt_seq/stmt_seq_macho.exe differ
diff --git a/llvm/test/tools/dsymutil/Inputs/private/tmp/stmt_seq/stmt_seq_macho.o b/llvm/test/tools/dsymutil/Inputs/private/tmp/stmt_seq/stmt_seq_macho.o
index 0da06940a023c..76fba6580055b 100644
Binary files a/llvm/test/tools/dsymutil/Inputs/private/tmp/stmt_seq/stmt_seq_macho.o and b/llvm/test/tools/dsymutil/Inputs/private/tmp/stmt_seq/stmt_seq_macho.o differ

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should avoid checking in binary files by using llvm/utils/update_test_body.py

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the DWARF linker - this is how all tests are currently: https://github.com/llvm/llvm-project/tree/main/llvm/test/tools/dsymutil/Inputs

So I just followed the existing pattern.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I just realized llvm/utils/update_test_body.py won't avoid checking in the binaries. But it does provide a standard way to generate these binaries. We should set a good example of using this new tool instead of ad hoc scripts.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there are 3 main approaches here:

  1. Follow existing pattern (check in binaries + script) - i.e. the current version of the PR
  2. Switch the test to fully use the update_test_body.py approach and instead of checking in binaries, check in the .yaml of the binary. (Example).
  3. Use a hybrid approach where we use update_test_body.py to generate the binaries to check in (I don't think there is currently any test using this method).

For this small fix I went with #1 but I am OK to change it to be either way. I'll let @JDevlieghere as the code owner to decide which approach is preferable.

Copy link
Member

@JDevlieghere JDevlieghere Mar 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't aware of update_test_body.py and I haven't tried it to generate Mach-Os for dsymutil test, but I'm happy to go with a standardized approach as I agree that the current ad-hoc approach is rather painful.

My preferred approach would be to use yaml for the object files (with yaml2obj) and for the debug map (which you can dump with dsymutil). I think that's the most readable/auditable, but it's also a pain to do by hand. Maybe the sweet spot is to have a script similar to update_test_body.py specifically for the DWARF linker?

Anyway, for this PR I think it's fine to stick with (1) as that's the established way of doing this while we settle on how to go forward. I always (try to) include instructions and source code to regenerate the test so maybe we can use some of the existing ones to test the approach, but that's definitely outside the scope of this PR.

@alx32 alx32 merged commit f1dad0b into llvm:main Mar 27, 2025
15 checks passed
@llvm-ci
Copy link
Collaborator

llvm-ci commented Mar 27, 2025

LLVM Buildbot has detected a new failure on builder lldb-aarch64-ubuntu running on linaro-lldb-aarch64-ubuntu while building llvm at step 6 "test".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/59/builds/14989

Here is the relevant piece of the build log for the reference
Step 6 (test) failure: build (failure)
...
PASS: lldb-api :: functionalities/stats_api/TestStatisticsAPI.py (585 of 2110)
PASS: lldb-api :: functionalities/reverse-execution/TestReverseContinueBreakpoints.py (586 of 2110)
PASS: lldb-api :: functionalities/gdb_remote_client/TestPlatformClient.py (587 of 2110)
PASS: lldb-api :: functionalities/step-avoids-regexp/TestStepAvoidsRegexp.py (588 of 2110)
PASS: lldb-api :: functionalities/signal/raise/TestRaise.py (589 of 2110)
PASS: lldb-api :: functionalities/step-avoids-no-debug/TestStepNoDebug.py (590 of 2110)
PASS: lldb-api :: functionalities/step_scripted/TestStepScripted.py (591 of 2110)
PASS: lldb-api :: functionalities/tail_call_frames/ambiguous_tail_call_seq1/TestAmbiguousTailCallSeq1.py (592 of 2110)
PASS: lldb-api :: functionalities/recursion/TestValueObjectRecursion.py (593 of 2110)
PASS: lldb-api :: functionalities/tail_call_frames/ambiguous_tail_call_seq2/TestAmbiguousTailCallSeq2.py (594 of 2110)
FAIL: lldb-api :: functionalities/stop-on-sharedlibrary-load/TestStopOnSharedlibraryEvents.py (595 of 2110)
******************** TEST 'lldb-api :: functionalities/stop-on-sharedlibrary-load/TestStopOnSharedlibraryEvents.py' FAILED ********************
Script:
--
/usr/bin/python3.10 /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/dotest.py -u CXXFLAGS -u CFLAGS --env LLVM_LIBS_DIR=/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/./lib --env LLVM_INCLUDE_DIR=/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/include --env LLVM_TOOLS_DIR=/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/./bin --arch aarch64 --build-dir /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/lldb-test-build.noindex --lldb-module-cache-dir /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/lldb-test-build.noindex/module-cache-lldb/lldb-api --clang-module-cache-dir /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/lldb-test-build.noindex/module-cache-clang/lldb-api --executable /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/./bin/lldb --compiler /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/./bin/clang --dsymutil /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/./bin/dsymutil --make /usr/bin/gmake --llvm-tools-dir /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/./bin --lldb-obj-root /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/tools/lldb --lldb-libs-dir /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/./lib /home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/llvm-project/lldb/test/API/functionalities/stop-on-sharedlibrary-load -p TestStopOnSharedlibraryEvents.py
--
Exit Code: -11

Command Output (stdout):
--
lldb version 21.0.0git (https://github.com/llvm/llvm-project.git revision f1dad0bcb58f2b8bf0d847d4a65909b797be4fa1)
  clang revision f1dad0bcb58f2b8bf0d847d4a65909b797be4fa1
  llvm revision f1dad0bcb58f2b8bf0d847d4a65909b797be4fa1
Skipping the following test categories: ['libc++', 'dsym', 'gmodules', 'debugserver', 'objc']

--
Command Output (stderr):
--
PASS: LLDB (/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/bin/clang-aarch64) :: test_auto_continue (TestStopOnSharedlibraryEvents.TestStopOnSharedlibraryEvents)
PASS: LLDB (/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/bin/clang-aarch64) :: test_continue_callback (TestStopOnSharedlibraryEvents.TestStopOnSharedlibraryEvents)
PASS: LLDB (/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/bin/clang-aarch64) :: test_failing_condition (TestStopOnSharedlibraryEvents.TestStopOnSharedlibraryEvents)
PASS: LLDB (/home/tcwg-buildbot/worker/lldb-aarch64-ubuntu/build/bin/clang-aarch64) :: test_stopping_breakpoints (TestStopOnSharedlibraryEvents.TestStopOnSharedlibraryEvents)
----------------------------------------------------------------------
Ran 4 tests in 2.011s

OK

--

********************
XFAIL: lldb-api :: functionalities/tail_call_frames/cross_dso/TestCrossDSOTailCalls.py (596 of 2110)
XFAIL: lldb-api :: functionalities/tail_call_frames/cross_object/TestCrossObjectTailCalls.py (597 of 2110)
PASS: lldb-api :: functionalities/step-vrs-interrupt/TestStepVrsInterruptTimeout.py (598 of 2110)
PASS: lldb-api :: functionalities/tail_call_frames/sbapi_support/TestTailCallFrameSBAPI.py (599 of 2110)
PASS: lldb-api :: functionalities/tail_call_frames/disambiguate_call_site/TestDisambiguateCallSite.py (600 of 2110)
PASS: lldb-api :: functionalities/tail_call_frames/disambiguate_paths_to_common_sink/TestDisambiguatePathsToCommonSink.py (601 of 2110)
PASS: lldb-api :: functionalities/tail_call_frames/disambiguate_tail_call_seq/TestDisambiguateTailCallSeq.py (602 of 2110)
PASS: lldb-api :: functionalities/tail_call_frames/inlining_and_tail_calls/TestInliningAndTailCalls.py (603 of 2110)
PASS: lldb-api :: functionalities/tail_call_frames/thread_step_out_or_return/TestSteppingOutWithArtificialFrames.py (604 of 2110)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants