Skip to content

[DebugInfo][DWARF] Add heapallocsite information #132073

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

thejh
Copy link
Contributor

@thejh thejh commented Mar 19, 2025

LLVM currently stores heapallocsite information in CodeView debuginfo, but not in DWARF debuginfo. Plumb it into DWARF as an LLVM-specific extension.

heapallocsite debug information is useful when it is combined with allocator instrumentation that stores caller addresses; I've used a previous version of this patch for:

  • analyzing memory usage by object type
  • analyzing the distributions of values of class members

Other possible uses might be:

  • attributing memory access profiles (for example, on Intel CPUs, from PEBS records with Linear Data Address) to object types or specific object members
  • adding type information to crash/ASAN reports

@llvmbot
Copy link
Member

llvmbot commented Mar 19, 2025

@llvm/pr-subscribers-debuginfo

@llvm/pr-subscribers-llvm-binary-utilities

Author: Jann (thejh)

Changes

LLVM currently stores heapallocsite information in CodeView debuginfo, but not in DWARF debuginfo. Plumb it into DWARF as an LLVM-specific extension.

heapallocsite debug information is useful when it is combined with allocator instrumentation that stores caller addresses; I've used a previous version of this patch for:

  • analyzing memory usage by object type
  • analyzing the distributions of values of class members

Other possible uses might be:

  • attributing memory access profiles (for example, on Intel CPUs, from PEBS records with Linear Data Address) to object types or specific object members
  • adding type information to crash/ASAN reports

Full diff: https://github.com/llvm/llvm-project/pull/132073.diff

6 Files Affected:

  • (modified) llvm/include/llvm/BinaryFormat/Dwarf.def (+1)
  • (modified) llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.cpp (+9-7)
  • (modified) llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.h (+2-1)
  • (modified) llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp (+13-12)
  • (modified) llvm/lib/DWARFLinker/Classic/DWARFLinker.cpp (+1)
  • (added) llvm/test/DebugInfo/X86/DW_AT_LLVM_heapallocsite.ll (+33)
diff --git a/llvm/include/llvm/BinaryFormat/Dwarf.def b/llvm/include/llvm/BinaryFormat/Dwarf.def
index e52324a8ebc12..51660b30568a5 100644
--- a/llvm/include/llvm/BinaryFormat/Dwarf.def
+++ b/llvm/include/llvm/BinaryFormat/Dwarf.def
@@ -624,6 +624,7 @@ HANDLE_DW_AT(0x3e09, LLVM_ptrauth_authenticates_null_values, 0, LLVM)
 HANDLE_DW_AT(0x3e0a, LLVM_ptrauth_authentication_mode, 0, LLVM)
 HANDLE_DW_AT(0x3e0b, LLVM_num_extra_inhabitants, 0, LLVM)
 HANDLE_DW_AT(0x3e0c, LLVM_stmt_sequence, 0, LLVM)
+HANDLE_DW_AT(0x3e0d, LLVM_heapallocsite, 0, LLVM)
 
 // Apple extensions.
 
diff --git a/llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.cpp b/llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.cpp
index dad14b98d2c3d..f14742d048c39 100644
--- a/llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.cpp
+++ b/llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.cpp
@@ -1289,12 +1289,10 @@ DwarfCompileUnit::getDwarf5OrGNULocationAtom(dwarf::LocationAtom Loc) const {
   }
 }
 
-DIE &DwarfCompileUnit::constructCallSiteEntryDIE(DIE &ScopeDIE,
-                                                 const DISubprogram *CalleeSP,
-                                                 bool IsTail,
-                                                 const MCSymbol *PCAddr,
-                                                 const MCSymbol *CallAddr,
-                                                 unsigned CallReg) {
+DIE &DwarfCompileUnit::constructCallSiteEntryDIE(
+    DIE &ScopeDIE, const DISubprogram *CalleeSP, bool IsTail,
+    const MCSymbol *PCAddr, const MCSymbol *CallAddr, unsigned CallReg,
+    DIType *AllocSiteTy) {
   // Insert a call site entry DIE within ScopeDIE.
   DIE &CallSiteDIE = createAndAddDIE(getDwarf5OrGNUTag(dwarf::DW_TAG_call_site),
                                      ScopeDIE, nullptr);
@@ -1303,7 +1301,7 @@ DIE &DwarfCompileUnit::constructCallSiteEntryDIE(DIE &ScopeDIE,
     // Indirect call.
     addAddress(CallSiteDIE, getDwarf5OrGNUAttr(dwarf::DW_AT_call_target),
                MachineLocation(CallReg));
-  } else {
+  } else if (CalleeSP) {
     DIE *CalleeDIE = getOrCreateSubprogramDIE(CalleeSP);
     assert(CalleeDIE && "Could not create DIE for call site entry origin");
     if (AddLinkageNamesToDeclCallOriginsForTuning(DD) &&
@@ -1348,6 +1346,10 @@ DIE &DwarfCompileUnit::constructCallSiteEntryDIE(DIE &ScopeDIE,
                     getDwarf5OrGNUAttr(dwarf::DW_AT_call_return_pc), PCAddr);
   }
 
+  if (AllocSiteTy) {
+    addType(CallSiteDIE, AllocSiteTy, dwarf::DW_AT_LLVM_heapallocsite);
+  }
+
   return CallSiteDIE;
 }
 
diff --git a/llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.h b/llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.h
index 104039db03c7c..52186d29c8272 100644
--- a/llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.h
+++ b/llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.h
@@ -285,7 +285,8 @@ class DwarfCompileUnit final : public DwarfUnit {
   /// the \p CallReg is set to 0.
   DIE &constructCallSiteEntryDIE(DIE &ScopeDIE, const DISubprogram *CalleeSP,
                                  bool IsTail, const MCSymbol *PCAddr,
-                                 const MCSymbol *CallAddr, unsigned CallReg);
+                                 const MCSymbol *CallAddr, unsigned CallReg,
+                                 DIType *AllocSiteTy);
   /// Construct call site parameter DIEs for the \p CallSiteDIE. The \p Params
   /// were collected by the \ref collectCallSiteParameters.
   /// Note: The order of parameters does not matter, since debuggers recognize
diff --git a/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp b/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
index b7b083028ef65..62ffe6e6e6c38 100644
--- a/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
+++ b/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
@@ -930,28 +930,29 @@ void DwarfDebug::constructCallSiteEntryDIEs(const DISubprogram &SP,
       if (MI.hasDelaySlot() && !delaySlotSupported(*&MI))
         return;
 
+      DIType *AllocSiteTy = cast_or_null<DIType>(MI.getHeapAllocMarker());
+
       // If this is a direct call, find the callee's subprogram.
       // In the case of an indirect call find the register that holds
       // the callee.
       const MachineOperand &CalleeOp = TII->getCalleeOperand(MI);
-      if (!CalleeOp.isGlobal() &&
-          (!CalleeOp.isReg() || !CalleeOp.getReg().isPhysical()))
-        continue;
 
       unsigned CallReg = 0;
       const DISubprogram *CalleeSP = nullptr;
       const Function *CalleeDecl = nullptr;
-      if (CalleeOp.isReg()) {
-        CallReg = CalleeOp.getReg();
-        if (!CallReg)
-          continue;
-      } else {
+      if (CalleeOp.isReg() && CalleeOp.getReg().isPhysical()) {
+        CallReg = CalleeOp.getReg(); // might be zero
+      } else if (CalleeOp.isGlobal()) {
         CalleeDecl = dyn_cast<Function>(CalleeOp.getGlobal());
-        if (!CalleeDecl || !CalleeDecl->getSubprogram())
-          continue;
-        CalleeSP = CalleeDecl->getSubprogram();
+        if (CalleeDecl)
+          CalleeSP = CalleeDecl->getSubprogram(); // might be nullptr
       }
 
+      // Omit DIE if we can't tell where the call goes *and* we don't want to
+      // add metadata to it.
+      if (CalleeSP == nullptr && CallReg == 0 && AllocSiteTy == nullptr)
+        continue;
+
       // TODO: Omit call site entries for runtime calls (objc_msgSend, etc).
 
       bool IsTail = TII->isTailCall(MI);
@@ -986,7 +987,7 @@ void DwarfDebug::constructCallSiteEntryDIEs(const DISubprogram &SP,
                         << (IsTail ? " [IsTail]" : "") << "\n");
 
       DIE &CallSiteDIE = CU.constructCallSiteEntryDIE(
-          ScopeDIE, CalleeSP, IsTail, PCAddr, CallAddr, CallReg);
+          ScopeDIE, CalleeSP, IsTail, PCAddr, CallAddr, CallReg, AllocSiteTy);
 
       // Optionally emit call-site-param debug info.
       if (emitDebugEntryValues()) {
diff --git a/llvm/lib/DWARFLinker/Classic/DWARFLinker.cpp b/llvm/lib/DWARFLinker/Classic/DWARFLinker.cpp
index f66773ad2e694..586d1c2abd5e0 100644
--- a/llvm/lib/DWARFLinker/Classic/DWARFLinker.cpp
+++ b/llvm/lib/DWARFLinker/Classic/DWARFLinker.cpp
@@ -109,6 +109,7 @@ static bool isODRAttribute(uint16_t Attr) {
   case dwarf::DW_AT_specification:
   case dwarf::DW_AT_abstract_origin:
   case dwarf::DW_AT_import:
+  case dwarf::DW_AT_LLVM_heapallocsite:
     return true;
   }
   llvm_unreachable("Improper attribute.");
diff --git a/llvm/test/DebugInfo/X86/DW_AT_LLVM_heapallocsite.ll b/llvm/test/DebugInfo/X86/DW_AT_LLVM_heapallocsite.ll
new file mode 100644
index 0000000000000..53975e02acbc3
--- /dev/null
+++ b/llvm/test/DebugInfo/X86/DW_AT_LLVM_heapallocsite.ll
@@ -0,0 +1,33 @@
+; RUN: llc -O3 -o %t -filetype=obj %s
+; RUN: llvm-dwarfdump %t | FileCheck %s
+
+; based on clang++ output for `int *alloc_int() { return new int; }`
+
+
+target triple = "x86_64-unknown-linux-gnu"
+
+define dso_local ptr @alloc_int() !dbg !3 {
+; CHECK: DW_TAG_subprogram
+entry:
+  %call = call ptr @alloc(i64 noundef 4), !heapallocsite !7
+; CHECK: DW_TAG_GNU_call_site
+; CHECK: DW_AT_LLVM_heapallocsite ([[ALLOCSITE:.*]])
+  ret ptr %call
+}
+
+; CHECK: {{.*}}[[ALLOCSITE]]: DW_TAG_base_type
+; CHECK: DW_AT_name ("int")
+
+declare dso_local ptr @alloc(i64 noundef)
+
+!llvm.dbg.cu = !{!0}
+!llvm.module.flags = !{!2}
+
+!0 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus_14, file: !1, emissionKind: FullDebug)
+!1 = !DIFile(filename: "a.cpp", directory: "/")
+!2 = !{i32 2, !"Debug Info Version", i32 3}
+!3 = distinct !DISubprogram(name: "alloc_int", scope: !1, file: !1, line: 1, type: !4, scopeLine: 1, flags: DIFlagPrototyped | DIFlagAllCallsDescribed, spFlags: DISPFlagDefinition, unit: !0)
+!4 = !DISubroutineType(types: !5)
+!5 = !{!6}
+!6 = !DIDerivedType(tag: DW_TAG_pointer_type, baseType: !7, size: 64)
+!7 = !DIBasicType(name: "int", size: 32, encoding: DW_ATE_signed)

@thejh
Copy link
Contributor Author

thejh commented Apr 3, 2025

ping - did I create this pull request correctly, or should I be CCing someone other than @JDevlieghere ?

@jryans jryans requested review from jmorse and dwblaikie April 3, 2025 15:20
@jryans
Copy link
Member

jryans commented Apr 3, 2025

ping - did I create this pull request correctly, or should I be CCing someone other than @JDevlieghere ?

I believe people are just generally overstretched in this area... I've added a few others who may have opinions here as well.

@dwblaikie
Copy link
Collaborator

Got any size data (like a bloaty before/after comparison of the clang binary with/without this patch?)?
& do you have a specific use case planned/coming along for this currently? Or is it more speculative?

LLVM currently stores heapallocsite information in CodeView debuginfo, but
not in DWARF debuginfo. heapallocsite information is useful for profiling
and memory analysis; plumb it into DWARF as an LLVM-specific extension.
@thejh thejh force-pushed the dwarf-heapallocsite-basic branch from 775c75a to 2674248 Compare April 3, 2025 18:57
@thejh
Copy link
Contributor Author

thejh commented Apr 3, 2025

@dwblaikie You mean a size comparison of clang built with debuginfo at a specific revision, built once using unmodified clang and once using clang with this patch applied?

Here you go - I built clang twice at git commit 5475834737663c0da3c444fcf1c8ab5567c39136, once using original clang (from git commit 5475834737663c0da3c444fcf1c8ab5567c39136), once using modified clang with my commit applied, both times using -DCMAKE_BUILD_TYPE=RelWithDebInfo -DLLVM_ENABLE_PROJECTS=clang:

$ [...]/bloaty [...] [...]/llvmp-build-2-debug-modcompiler/bin/clang -- [...]/llvmp-build-2-debug-origcompiler/bin/clang
    FILE SIZE        VM SIZE
 --------------  --------------
  +0.0%  +256Ki  [ = ]       0    .debug_info
  +0.3% +94.2Ki  [ = ]       0    .debug_addr
  +0.2% +23.4Ki  [ = ]       0    .debug_abbrev
  +0.0% +4.85Ki  [ = ]       0    .debug_loclists
  +0.0% +2.63Ki  [ = ]       0    .debug_str_offsets
  +0.0% +1.45Ki  [ = ]       0    .debug_str
  +0.0% +1.03Ki  [ = ]       0    .debug_rnglists
  -0.0%      -2  [ = ]       0    .debug_line_str
  +0.0%  +384Ki  [ = ]       0    TOTAL
$ ls -Ll [...]/llvmp-build-2-debug-{orig,mod}compiler/bin/clang
-rwxr-x--- 1 [...] 1454958680 Apr  3 20:54 [...]/llvmp-build-2-debug-modcompiler/bin/clang
-rwxr-x--- 1 [...] 1454564968 Apr  3 19:05 [...]/llvmp-build-2-debug-origcompiler/bin/clang

Grepping the output of running llvm-dwarfdump on llvmp-build-2-debug-modcompiler/bin/clang, it looks like there are 17639 DW_AT_LLVM_heapallocsite DWARF entries.

If you think that's too much to add in by default when there is currently no tooling that can make use of this, I'd be happy to gate this on some new default-disabled flag.

Yes, I have a planned usecase - I would like to eventually use this feature for augmenting Linux kernel memory access traces, and maybe also ASAN crash reports (which already report allocator call site addresses), with type information. However, that usecase requires more LLVM/clang patches on top of this, since Linux kernel code uses something like malloc(sizeof(<type>)) rather than C++ new. I have not yet talked about this with upstream Linux developers, and figured I'd maybe simultaneously talk about this idea on the Linux mailing list and in https://discourse.llvm.org/ once I have some draft patch(es) for handling some more complicated versions of patterns like malloc(sizeof()).

Another usecase I had in the past, for which I originally wrote this patch, was to analyze heap memory usage of a C++ codebase (like seeing which classes use how much memory, or seeing statistics for what the most common values of some class member are) - for that usecase, this patch alone mostly did the job (combined with allocator changes to record callsites). Since this patch was already independently useful for that kind of usecase, I figured I might as well land this first.
(And since https://llvm.org/docs/Contributing.html#how-to-submit-a-patch says that the preferred way of sending patches is to send one commit at a time, I figured you might also prefer getting piecemeal patches as long as they're independently useful.)

@thejh
Copy link
Contributor Author

thejh commented Apr 3, 2025

For that old usecase, the tooling I built on top of DWARF heapallocsite debuginfo was able to print heap analysis like this: https://gist.github.com/thejh/92befd69a727b02aa4262075c554a872

That shows heap allocations grouped by allocation type, showing the three most common values of each member along with how often that value occurs.

@pinskia
Copy link

pinskia commented Apr 3, 2025

Does it make sense to propose the extension first to dwarf standard (https://dwarfstd.org/) before adding it to LLVM?

@thejh
Copy link
Contributor Author

thejh commented Apr 3, 2025

@pinskia I'm not familiar with how DWARF standard development works in practice - do you mean just propose it for the DWARF standard, then add it to LLVM as an LLVM extension in parallel while the DWARF committee figures out whether they want to standardize it? Or actually land it in the (working draft? official?) DWARF standard before adding the feature to LLVM at all?

@pinskia
Copy link

pinskia commented Apr 4, 2025

@pinskia I'm not familiar with how DWARF standard development works in practice - do you mean just propose it for the DWARF standard, then add it to LLVM as an LLVM extension in parallel while the DWARF committee figures out whether they want to standardize it? Or actually land it in the (working draft? official?) DWARF standard before adding the feature to LLVM at all?

First start with https://dwarfstd.org/procedures.html .
Second having a full written up proposal of the extension is useful NOT just for the dwarf standard but also for others to consume/emit the extension and not just based on the code in LLVM. For me having a decent written up proposal shoud be enough for adding the feature to any emittor/consumer. I don't know how this was handled before in LLVM but since this will be consumed by more than LLVM/LLDB, having a proposal at least documenting the extension more than just a number should be a good thing to do anyways.

@pogo59
Copy link
Collaborator

pogo59 commented Apr 4, 2025

The new attribute should be documented. It doesn't have to be proposed to the DWARF committee, but that would be a good idea if anyone wants to implement the same feature in another compiler. I take it there's no gcc equivalent?

@thejh
Copy link
Contributor Author

thejh commented Apr 4, 2025

@pinskia Sounds reasonable, I'll try to write and submit a proposal.

@thejh
Copy link
Contributor Author

thejh commented Apr 7, 2025

I have now submitted a DWARF enhancement proposal to dwarf-discuss (but an issue number has not yet been assigned): https://lists.dwarfstd.org/pipermail/dwarf-discuss/2025-April/002656.html

I take it there's no gcc equivalent?

I am not aware of one.

@thejh
Copy link
Contributor Author

thejh commented Apr 8, 2025

The proposal is now listed at https://dwarfstd.org/issues/250407.1.html .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants