[KeyIntsr][Clang] Builtins alloca auto-init atom

[OCHyams]

[KeyIntsr][Clang] Builtins alloca auto-init atom

This patch is part of a stack that teaches Clang to generate Key Instructions
metadata for C and C++.

The Key Instructions project is introduced, including a "quick summary" section
at the top which adds context for this PR, here:
https://discourse.llvm.org/t/rfc-improving-is-stmt-placement-for-better-interactive-debugging/82668

The feature is only functional in LLVM if LLVM is built with CMake flag
LLVM_EXPERIMENTAL_KEY_INSTRUCTIONs. Eventually that flag will be removed.

The Clang-side work is demoed here:
#130943

[KeyInstr][Clang] matrix store atom

[KeyInstr][Clang] Store-like builtin atoms

[OCHyams]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• [NFC][KeyInstr][Clang] Add some additional tests #134654
• [KeyInstr][Clang] Coerced store atoms #134653
• [KeyInstr][Clang] Ret atom #134652
• [KeyIntsr][Clang] Builtins alloca auto-init atom #134651 👈 (View in Graphite)
• [KeyInstr][Clang] Assign matrix element atom #134650
• [KeyInstr][Clang] Assign vector element atom #134649
• [KeyInstr][Clang] Bitfield atom #134648
• [KeyInstr][Clang] For range stmt atoms #134647
• [KeyInstr][Clang] For stmt atom #134646
• [KeyInstr][Clang] While stmt atom #134645
• [KeyInstr][Clang] Do stmt atom #134644
• [KeyInstr][Clang] Switch stmt atom #134643
• [KeyInstr][Clang] If stmt atom #134642
• [KeyInstr][Clang] Catch variable init atom #134641
• [KeyInstr][Clang] Member initalization atom #134640
• [KeyInstr][Clang] Aggregate init + copy #134639
• [KeyInstr] Complex assignment atoms #134638
• [KeyInstr][Clang] Assignment atom group #134637
• [KeyInstr][Clang] Static variable init atom #134636
• [KeyInstr][Clang] Agg init atom #134635
• [Clang][NFC] Move some static functions into CodeGenFunction #134634
• [KeyInstr][Clang] Scalar init atom #134633
• [KeyInstr][Clang] Add ApplyAtomGroup #134632
• [KeyInstr][Clang] Add Clang option -g[no-]key-instructions #134627
• [KeyInstr][DwarfDebug] Add is_stmt emission support #133495
• [KeyInstr] Add MIR parser support #133494
• [KeyInstr] Remap cloned PHIs in BreakCriticalEdges #133493
• Propagate DebugLocs on phis in BreakCriticalEdges #133492
• [KeyInstr][LoopUnswitch] Remap cloned instructions' atoms #133491
• [KeyInstr][LoopRotate] Remap atoms of duplicated instructions #133490
• [KeyInstr][LoopUnroll] Remap atoms while unrolling #133489
• [KeyInstr][JumpThreading] Remap atoms duping bb with cond br on phi into pred #133488
• [KeyInstr][JumpThreading] Remap atoms after threading #133487
• [KeyInstr][JumpThreading] Remap atoms in blocks duplicated for threading #133486
• [KeyInstr][Inline] Don't propagate atoms to inlined nodebug instructions #133485
• [KeyInstr][SimplifyCFG] Remap atoms after duplication for threading #133484
• [KeyInstr][debugify] Add --debugify-atoms to add key instructions metadata #133483
• [KeyInstr][SimplifyCFG] Remap atoms when folding br to common succ into pred #133482
• [KeyInstr] Inline atom info #133481
• [KeyInstr] Merge atoms in DILocation::getMergedLocation #133480
• [NFC][KeyInstr] Add Atom Group (re)mapping #133479
• [KeyInstr] Add Atom Group waterline to LLVMContext #133478
• [KeyInstr] Add fields to DILocation behind compile time flag #133477
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[llvmbot]

@llvm/pr-subscribers-clang

Author: Orlando Cazalet-Hyams (OCHyams)

Changes

[KeyIntsr][Clang] Builtins alloca auto-init atom

This patch is part of a stack that teaches Clang to generate Key Instructions
metadata for C and C++.

The Key Instructions project is introduced, including a "quick summary" section
at the top which adds context for this PR, here:
https://discourse.llvm.org/t/rfc-improving-is-stmt-placement-for-better-interactive-debugging/82668

The feature is only functional in LLVM if LLVM is built with CMake flag
LLVM_EXPERIMENTAL_KEY_INSTRUCTIONs. Eventually that flag will be removed.

The Clang-side work is demoed here:
#130943

[KeyInstr][Clang] matrix store atom

[KeyInstr][Clang] Store-like builtin atoms

---

Full diff: https://github.com/llvm/llvm-project/pull/134651.diff

2 Files Affected:

• (modified) clang/lib/CodeGen/CGBuiltin.cpp (+23-10)
• (added) clang/test/KeyInstructions/builtin.c (+77)

diff --git a/clang/lib/CodeGen/CGBuiltin.cpp b/clang/lib/CodeGen/CGBuiltin.cpp
index 91ac7c5847b02..a735d5fa151ac 100644
--- a/clang/lib/CodeGen/CGBuiltin.cpp
+++ b/clang/lib/CodeGen/CGBuiltin.cpp
@@ -29,6 +29,7 @@
 #include "clang/Basic/TargetInfo.h"
 #include "clang/Basic/TargetOptions.h"
 #include "clang/Frontend/FrontendDiagnostic.h"
+#include "llvm/ADT/ScopeExit.h"
 #include "llvm/IR/InlineAsm.h"
 #include "llvm/IR/Intrinsics.h"
 #include "llvm/IR/IntrinsicsX86.h"
@@ -138,6 +139,7 @@ static void initializeAlloca(CodeGenFunction &CGF, AllocaInst *AI, Value *Size,
   if (CGF.CGM.stopAutoInit())
     return;
   auto *I = CGF.Builder.CreateMemSet(AI, Byte, Size, AlignmentInBytes);
+  CGF.addInstToCurrentSourceAtom(I, nullptr);
   I->addAnnotationMetadata("auto-init");
 }
 
@@ -3937,6 +3939,7 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
         Matrix, Dst.emitRawPointer(*this),
         Align(Dst.getAlignment().getQuantity()), Stride, IsVolatile,
         MatrixTy->getNumRows(), MatrixTy->getNumColumns());
+    addInstToNewSourceAtom(cast<llvm::Instruction>(Result), nullptr);
     return RValue::get(Result);
   }
 
@@ -4097,7 +4100,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
     Value *SizeVal = EmitScalarExpr(E->getArg(1));
     EmitNonNullArgCheck(Dest, E->getArg(0)->getType(),
                         E->getArg(0)->getExprLoc(), FD, 0);
-    Builder.CreateMemSet(Dest, Builder.getInt8(0), SizeVal, false);
+    auto *I = Builder.CreateMemSet(Dest, Builder.getInt8(0), SizeVal, false);
+    addInstToNewSourceAtom(I, nullptr);
     return RValue::get(nullptr);
   }
 
@@ -4112,7 +4116,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
     EmitNonNullArgCheck(RValue::get(Dest.emitRawPointer(*this)),
                         E->getArg(1)->getType(), E->getArg(1)->getExprLoc(), FD,
                         0);
-    Builder.CreateMemMove(Dest, Src, SizeVal, false);
+    auto *I = Builder.CreateMemMove(Dest, Src, SizeVal, false);
+    addInstToNewSourceAtom(I, nullptr);
     return RValue::get(nullptr);
   }
 
@@ -4125,7 +4130,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
     Value *SizeVal = EmitScalarExpr(E->getArg(2));
     EmitArgCheck(TCK_Store, Dest, E->getArg(0), 0);
     EmitArgCheck(TCK_Load, Src, E->getArg(1), 1);
-    Builder.CreateMemCpy(Dest, Src, SizeVal, false);
+    auto *I = Builder.CreateMemCpy(Dest, Src, SizeVal, false);
+    addInstToNewSourceAtom(I, nullptr);
     if (BuiltinID == Builtin::BImempcpy ||
         BuiltinID == Builtin::BI__builtin_mempcpy)
       return RValue::get(Builder.CreateInBoundsGEP(
@@ -4141,7 +4147,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
         E->getArg(2)->EvaluateKnownConstInt(getContext()).getZExtValue();
     EmitArgCheck(TCK_Store, Dest, E->getArg(0), 0);
     EmitArgCheck(TCK_Load, Src, E->getArg(1), 1);
-    Builder.CreateMemCpyInline(Dest, Src, Size);
+    auto *I = Builder.CreateMemCpyInline(Dest, Src, Size);
+    addInstToNewSourceAtom(I, nullptr);
     return RValue::get(nullptr);
   }
 
@@ -4162,7 +4169,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
     Address Dest = EmitPointerWithAlignment(E->getArg(0));
     Address Src = EmitPointerWithAlignment(E->getArg(1));
     Value *SizeVal = llvm::ConstantInt::get(Builder.getContext(), Size);
-    Builder.CreateMemCpy(Dest, Src, SizeVal, false);
+    auto *I = Builder.CreateMemCpy(Dest, Src, SizeVal, false);
+    addInstToNewSourceAtom(I, nullptr);
     return RValue::get(Dest, *this);
   }
 
@@ -4188,7 +4196,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
     Address Dest = EmitPointerWithAlignment(E->getArg(0));
     Address Src = EmitPointerWithAlignment(E->getArg(1));
     Value *SizeVal = llvm::ConstantInt::get(Builder.getContext(), Size);
-    Builder.CreateMemMove(Dest, Src, SizeVal, false);
+    auto *I = Builder.CreateMemMove(Dest, Src, SizeVal, false);
+    addInstToNewSourceAtom(I, nullptr);
     return RValue::get(Dest, *this);
   }
 
@@ -4199,7 +4208,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
     Value *SizeVal = EmitScalarExpr(E->getArg(2));
     EmitArgCheck(TCK_Store, Dest, E->getArg(0), 0);
     EmitArgCheck(TCK_Load, Src, E->getArg(1), 1);
-    Builder.CreateMemMove(Dest, Src, SizeVal, false);
+    auto *I = Builder.CreateMemMove(Dest, Src, SizeVal, false);
+    addInstToNewSourceAtom(I, nullptr);
     return RValue::get(Dest, *this);
   }
   case Builtin::BImemset:
@@ -4210,7 +4220,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
     Value *SizeVal = EmitScalarExpr(E->getArg(2));
     EmitNonNullArgCheck(Dest, E->getArg(0)->getType(),
                         E->getArg(0)->getExprLoc(), FD, 0);
-    Builder.CreateMemSet(Dest, ByteVal, SizeVal, false);
+    auto *I = Builder.CreateMemSet(Dest, ByteVal, SizeVal, false);
+    addInstToNewSourceAtom(I, nullptr);
     return RValue::get(Dest, *this);
   }
   case Builtin::BI__builtin_memset_inline: {
@@ -4222,7 +4233,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
     EmitNonNullArgCheck(RValue::get(Dest.emitRawPointer(*this)),
                         E->getArg(0)->getType(), E->getArg(0)->getExprLoc(), FD,
                         0);
-    Builder.CreateMemSetInline(Dest, ByteVal, Size);
+    auto *I = Builder.CreateMemSetInline(Dest, ByteVal, Size);
+    addInstToNewSourceAtom(I, nullptr);
     return RValue::get(nullptr);
   }
   case Builtin::BI__builtin___memset_chk: {
@@ -4239,7 +4251,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
     Value *ByteVal = Builder.CreateTrunc(EmitScalarExpr(E->getArg(1)),
                                          Builder.getInt8Ty());
     Value *SizeVal = llvm::ConstantInt::get(Builder.getContext(), Size);
-    Builder.CreateMemSet(Dest, ByteVal, SizeVal, false);
+    auto *I = Builder.CreateMemSet(Dest, ByteVal, SizeVal, false);
+    addInstToNewSourceAtom(I, nullptr);
     return RValue::get(Dest, *this);
   }
   case Builtin::BI__builtin_wmemchr: {
diff --git a/clang/test/KeyInstructions/builtin.c b/clang/test/KeyInstructions/builtin.c
new file mode 100644
index 0000000000000..5129a4ac2c482
--- /dev/null
+++ b/clang/test/KeyInstructions/builtin.c
@@ -0,0 +1,77 @@
+
+// RUN: %clang -gkey-instructions -x c++ %s -gmlt -gno-column-info -S -emit-llvm -o - -ftrivial-auto-var-init=zero -fenable-matrix -Xclang -disable-llvm-passes \
+// RUN: | FileCheck %s --implicit-check-not atomGroup --implicit-check-not atomRank
+
+// RUN: %clang -gkey-instructions -x c %s -gmlt -gno-column-info -S -emit-llvm -o - -ftrivial-auto-var-init=zero -fenable-matrix -Xclang -disable-llvm-passes \
+// RUN: | FileCheck %s --implicit-check-not atomGroup --implicit-check-not atomRank
+
+typedef float m2x2 __attribute__((matrix_type(2, 2)));
+m2x2 mat;
+float f4[4];
+float f8[8];
+
+void fun() {
+// CHECK: %a = alloca ptr, align 8
+// CHECK: %0 = alloca i8, i64 4{{.*}}, !dbg [[G1R2:!.*]]
+// CHECK: call void @llvm.memset{{.*}}, !dbg [[G1R1:!.*]], !annotation
+// CHECK: store ptr %0, ptr %a{{.*}}, !dbg [[G1R1:!.*]]
+    void *a = __builtin_alloca(4);
+
+// CHECK: %1 = alloca i8, i64 4{{.*}}, !dbg [[G2R2:!.*]]
+// CHECK: call void @llvm.memset{{.*}}, !dbg [[G2R1:!.*]], !annotation
+// CHECK: store ptr %1, ptr %b{{.*}}, !dbg [[G2R1:!.*]]
+    void *b = __builtin_alloca_with_align(4, 8);
+
+// CHECK: call void @llvm.matrix.column.major.store.v4f32{{.*}}, !dbg [[G3R1:!.*]]
+    __builtin_matrix_column_major_store(mat, f4, sizeof(float) * 2);
+
+// CHECK: call void @llvm.memset{{.*}}, !dbg [[G4R1:!.*]]
+    __builtin_bzero(f4, sizeof(float) * 2);
+
+// CHECK: call void @llvm.memmove{{.*}}, !dbg [[G5R1:!.*]]
+    __builtin_bcopy(f4, f8, sizeof(float) * 4);
+
+// CHECK: call void @llvm.memcpy{{.*}}, !dbg [[G6R1:!.*]]
+    __builtin_memcpy(f4, f8, sizeof(float) * 4);
+
+// CHECK: call void @llvm.memcpy{{.*}}, !dbg [[G7R1:!.*]]
+    __builtin_mempcpy(f4, f8, sizeof(float) * 4);
+
+// CHECK: call void @llvm.memcpy{{.*}}, !dbg [[G8R1:!.*]]
+    __builtin_memcpy_inline(f4, f8, sizeof(float) * 4);
+
+// CHECK: call void @llvm.memcpy{{.*}}, !dbg [[G9R1:!.*]]
+    __builtin___memcpy_chk(f4, f8, sizeof(float) * 4, -1);
+
+// CHECK: call void @llvm.memmove{{.*}}, !dbg [[G10R1:!.*]]
+    __builtin___memmove_chk(f4, f8, sizeof(float) * 4, -1);
+
+// CHECK: call void @llvm.memmove{{.*}}, !dbg [[G11R1:!.*]]
+    __builtin_memmove(f4, f8, sizeof(float) * 4);
+
+// CHECK: call void @llvm.memset{{.*}}, !dbg [[G12R1:!.*]]
+    __builtin_memset(f4, 0, sizeof(float) * 4);
+
+// CHECK: call void @llvm.memset{{.*}}, !dbg [[G13R1:!.*]]
+    __builtin_memset_inline(f4, 0, sizeof(float) * 4);
+
+// CHECK: call void @llvm.memset{{.*}}, !dbg [[G14R1:!.*]]
+    __builtin___memset_chk(f4, 0, sizeof(float), -1);
+}
+
+// CHECK: [[G1R2]] = !DILocation({{.*}}, atomGroup: 1, atomRank: 2)
+// CHECK: [[G1R1]] = !DILocation({{.*}}, atomGroup: 1, atomRank: 1)
+// CHECK: [[G2R2]] = !DILocation({{.*}}, atomGroup: 2, atomRank: 2)
+// CHECK: [[G2R1]] = !DILocation({{.*}}, atomGroup: 2, atomRank: 1)
+// CHECK: [[G3R1]] = !DILocation({{.*}}, atomGroup: 3, atomRank: 1)
+// CHECK: [[G4R1]] = !DILocation({{.*}}, atomGroup: 4, atomRank: 1)
+// CHECK: [[G5R1]] = !DILocation({{.*}}, atomGroup: 5, atomRank: 1)
+// CHECK: [[G6R1]] = !DILocation({{.*}}, atomGroup: 6, atomRank: 1)
+// CHECK: [[G7R1]] = !DILocation({{.*}}, atomGroup: 7, atomRank: 1)
+// CHECK: [[G8R1]] = !DILocation({{.*}}, atomGroup: 8, atomRank: 1)
+// CHECK: [[G9R1]] = !DILocation({{.*}}, atomGroup: 9, atomRank: 1)
+// CHECK: [[G10R1]] = !DILocation({{.*}}, atomGroup: 10, atomRank: 1)
+// CHECK: [[G11R1]] = !DILocation({{.*}}, atomGroup: 11, atomRank: 1)
+// CHECK: [[G12R1]] = !DILocation({{.*}}, atomGroup: 12, atomRank: 1)
+// CHECK: [[G13R1]] = !DILocation({{.*}}, atomGroup: 13, atomRank: 1)
+// CHECK: [[G14R1]] = !DILocation({{.*}}, atomGroup: 14, atomRank: 1)

[llvmbot]

@llvm/pr-subscribers-clang-codegen

Author: Orlando Cazalet-Hyams (OCHyams)

Changes

[KeyIntsr][Clang] Builtins alloca auto-init atom

This patch is part of a stack that teaches Clang to generate Key Instructions
metadata for C and C++.

The Key Instructions project is introduced, including a "quick summary" section
at the top which adds context for this PR, here:
https://discourse.llvm.org/t/rfc-improving-is-stmt-placement-for-better-interactive-debugging/82668

The feature is only functional in LLVM if LLVM is built with CMake flag
LLVM_EXPERIMENTAL_KEY_INSTRUCTIONs. Eventually that flag will be removed.

The Clang-side work is demoed here:
#130943

[KeyInstr][Clang] matrix store atom

[KeyInstr][Clang] Store-like builtin atoms

---

Full diff: https://github.com/llvm/llvm-project/pull/134651.diff

2 Files Affected:

• (modified) clang/lib/CodeGen/CGBuiltin.cpp (+23-10)
• (added) clang/test/KeyInstructions/builtin.c (+77)

diff --git a/clang/lib/CodeGen/CGBuiltin.cpp b/clang/lib/CodeGen/CGBuiltin.cpp
index 91ac7c5847b02..a735d5fa151ac 100644
--- a/clang/lib/CodeGen/CGBuiltin.cpp
+++ b/clang/lib/CodeGen/CGBuiltin.cpp
@@ -29,6 +29,7 @@
 #include "clang/Basic/TargetInfo.h"
 #include "clang/Basic/TargetOptions.h"
 #include "clang/Frontend/FrontendDiagnostic.h"
+#include "llvm/ADT/ScopeExit.h"
 #include "llvm/IR/InlineAsm.h"
 #include "llvm/IR/Intrinsics.h"
 #include "llvm/IR/IntrinsicsX86.h"
@@ -138,6 +139,7 @@ static void initializeAlloca(CodeGenFunction &CGF, AllocaInst *AI, Value *Size,
   if (CGF.CGM.stopAutoInit())
     return;
   auto *I = CGF.Builder.CreateMemSet(AI, Byte, Size, AlignmentInBytes);
+  CGF.addInstToCurrentSourceAtom(I, nullptr);
   I->addAnnotationMetadata("auto-init");
 }
 
@@ -3937,6 +3939,7 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
         Matrix, Dst.emitRawPointer(*this),
         Align(Dst.getAlignment().getQuantity()), Stride, IsVolatile,
         MatrixTy->getNumRows(), MatrixTy->getNumColumns());
+    addInstToNewSourceAtom(cast<llvm::Instruction>(Result), nullptr);
     return RValue::get(Result);
   }
 
@@ -4097,7 +4100,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
     Value *SizeVal = EmitScalarExpr(E->getArg(1));
     EmitNonNullArgCheck(Dest, E->getArg(0)->getType(),
                         E->getArg(0)->getExprLoc(), FD, 0);
-    Builder.CreateMemSet(Dest, Builder.getInt8(0), SizeVal, false);
+    auto *I = Builder.CreateMemSet(Dest, Builder.getInt8(0), SizeVal, false);
+    addInstToNewSourceAtom(I, nullptr);
     return RValue::get(nullptr);
   }
 
@@ -4112,7 +4116,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
     EmitNonNullArgCheck(RValue::get(Dest.emitRawPointer(*this)),
                         E->getArg(1)->getType(), E->getArg(1)->getExprLoc(), FD,
                         0);
-    Builder.CreateMemMove(Dest, Src, SizeVal, false);
+    auto *I = Builder.CreateMemMove(Dest, Src, SizeVal, false);
+    addInstToNewSourceAtom(I, nullptr);
     return RValue::get(nullptr);
   }
 
@@ -4125,7 +4130,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
     Value *SizeVal = EmitScalarExpr(E->getArg(2));
     EmitArgCheck(TCK_Store, Dest, E->getArg(0), 0);
     EmitArgCheck(TCK_Load, Src, E->getArg(1), 1);
-    Builder.CreateMemCpy(Dest, Src, SizeVal, false);
+    auto *I = Builder.CreateMemCpy(Dest, Src, SizeVal, false);
+    addInstToNewSourceAtom(I, nullptr);
     if (BuiltinID == Builtin::BImempcpy ||
         BuiltinID == Builtin::BI__builtin_mempcpy)
       return RValue::get(Builder.CreateInBoundsGEP(
@@ -4141,7 +4147,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
         E->getArg(2)->EvaluateKnownConstInt(getContext()).getZExtValue();
     EmitArgCheck(TCK_Store, Dest, E->getArg(0), 0);
     EmitArgCheck(TCK_Load, Src, E->getArg(1), 1);
-    Builder.CreateMemCpyInline(Dest, Src, Size);
+    auto *I = Builder.CreateMemCpyInline(Dest, Src, Size);
+    addInstToNewSourceAtom(I, nullptr);
     return RValue::get(nullptr);
   }
 
@@ -4162,7 +4169,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
     Address Dest = EmitPointerWithAlignment(E->getArg(0));
     Address Src = EmitPointerWithAlignment(E->getArg(1));
     Value *SizeVal = llvm::ConstantInt::get(Builder.getContext(), Size);
-    Builder.CreateMemCpy(Dest, Src, SizeVal, false);
+    auto *I = Builder.CreateMemCpy(Dest, Src, SizeVal, false);
+    addInstToNewSourceAtom(I, nullptr);
     return RValue::get(Dest, *this);
   }
 
@@ -4188,7 +4196,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
     Address Dest = EmitPointerWithAlignment(E->getArg(0));
     Address Src = EmitPointerWithAlignment(E->getArg(1));
     Value *SizeVal = llvm::ConstantInt::get(Builder.getContext(), Size);
-    Builder.CreateMemMove(Dest, Src, SizeVal, false);
+    auto *I = Builder.CreateMemMove(Dest, Src, SizeVal, false);
+    addInstToNewSourceAtom(I, nullptr);
     return RValue::get(Dest, *this);
   }
 
@@ -4199,7 +4208,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
     Value *SizeVal = EmitScalarExpr(E->getArg(2));
     EmitArgCheck(TCK_Store, Dest, E->getArg(0), 0);
     EmitArgCheck(TCK_Load, Src, E->getArg(1), 1);
-    Builder.CreateMemMove(Dest, Src, SizeVal, false);
+    auto *I = Builder.CreateMemMove(Dest, Src, SizeVal, false);
+    addInstToNewSourceAtom(I, nullptr);
     return RValue::get(Dest, *this);
   }
   case Builtin::BImemset:
@@ -4210,7 +4220,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
     Value *SizeVal = EmitScalarExpr(E->getArg(2));
     EmitNonNullArgCheck(Dest, E->getArg(0)->getType(),
                         E->getArg(0)->getExprLoc(), FD, 0);
-    Builder.CreateMemSet(Dest, ByteVal, SizeVal, false);
+    auto *I = Builder.CreateMemSet(Dest, ByteVal, SizeVal, false);
+    addInstToNewSourceAtom(I, nullptr);
     return RValue::get(Dest, *this);
   }
   case Builtin::BI__builtin_memset_inline: {
@@ -4222,7 +4233,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
     EmitNonNullArgCheck(RValue::get(Dest.emitRawPointer(*this)),
                         E->getArg(0)->getType(), E->getArg(0)->getExprLoc(), FD,
                         0);
-    Builder.CreateMemSetInline(Dest, ByteVal, Size);
+    auto *I = Builder.CreateMemSetInline(Dest, ByteVal, Size);
+    addInstToNewSourceAtom(I, nullptr);
     return RValue::get(nullptr);
   }
   case Builtin::BI__builtin___memset_chk: {
@@ -4239,7 +4251,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
     Value *ByteVal = Builder.CreateTrunc(EmitScalarExpr(E->getArg(1)),
                                          Builder.getInt8Ty());
     Value *SizeVal = llvm::ConstantInt::get(Builder.getContext(), Size);
-    Builder.CreateMemSet(Dest, ByteVal, SizeVal, false);
+    auto *I = Builder.CreateMemSet(Dest, ByteVal, SizeVal, false);
+    addInstToNewSourceAtom(I, nullptr);
     return RValue::get(Dest, *this);
   }
   case Builtin::BI__builtin_wmemchr: {
diff --git a/clang/test/KeyInstructions/builtin.c b/clang/test/KeyInstructions/builtin.c
new file mode 100644
index 0000000000000..5129a4ac2c482
--- /dev/null
+++ b/clang/test/KeyInstructions/builtin.c
@@ -0,0 +1,77 @@
+
+// RUN: %clang -gkey-instructions -x c++ %s -gmlt -gno-column-info -S -emit-llvm -o - -ftrivial-auto-var-init=zero -fenable-matrix -Xclang -disable-llvm-passes \
+// RUN: | FileCheck %s --implicit-check-not atomGroup --implicit-check-not atomRank
+
+// RUN: %clang -gkey-instructions -x c %s -gmlt -gno-column-info -S -emit-llvm -o - -ftrivial-auto-var-init=zero -fenable-matrix -Xclang -disable-llvm-passes \
+// RUN: | FileCheck %s --implicit-check-not atomGroup --implicit-check-not atomRank
+
+typedef float m2x2 __attribute__((matrix_type(2, 2)));
+m2x2 mat;
+float f4[4];
+float f8[8];
+
+void fun() {
+// CHECK: %a = alloca ptr, align 8
+// CHECK: %0 = alloca i8, i64 4{{.*}}, !dbg [[G1R2:!.*]]
+// CHECK: call void @llvm.memset{{.*}}, !dbg [[G1R1:!.*]], !annotation
+// CHECK: store ptr %0, ptr %a{{.*}}, !dbg [[G1R1:!.*]]
+    void *a = __builtin_alloca(4);
+
+// CHECK: %1 = alloca i8, i64 4{{.*}}, !dbg [[G2R2:!.*]]
+// CHECK: call void @llvm.memset{{.*}}, !dbg [[G2R1:!.*]], !annotation
+// CHECK: store ptr %1, ptr %b{{.*}}, !dbg [[G2R1:!.*]]
+    void *b = __builtin_alloca_with_align(4, 8);
+
+// CHECK: call void @llvm.matrix.column.major.store.v4f32{{.*}}, !dbg [[G3R1:!.*]]
+    __builtin_matrix_column_major_store(mat, f4, sizeof(float) * 2);
+
+// CHECK: call void @llvm.memset{{.*}}, !dbg [[G4R1:!.*]]
+    __builtin_bzero(f4, sizeof(float) * 2);
+
+// CHECK: call void @llvm.memmove{{.*}}, !dbg [[G5R1:!.*]]
+    __builtin_bcopy(f4, f8, sizeof(float) * 4);
+
+// CHECK: call void @llvm.memcpy{{.*}}, !dbg [[G6R1:!.*]]
+    __builtin_memcpy(f4, f8, sizeof(float) * 4);
+
+// CHECK: call void @llvm.memcpy{{.*}}, !dbg [[G7R1:!.*]]
+    __builtin_mempcpy(f4, f8, sizeof(float) * 4);
+
+// CHECK: call void @llvm.memcpy{{.*}}, !dbg [[G8R1:!.*]]
+    __builtin_memcpy_inline(f4, f8, sizeof(float) * 4);
+
+// CHECK: call void @llvm.memcpy{{.*}}, !dbg [[G9R1:!.*]]
+    __builtin___memcpy_chk(f4, f8, sizeof(float) * 4, -1);
+
+// CHECK: call void @llvm.memmove{{.*}}, !dbg [[G10R1:!.*]]
+    __builtin___memmove_chk(f4, f8, sizeof(float) * 4, -1);
+
+// CHECK: call void @llvm.memmove{{.*}}, !dbg [[G11R1:!.*]]
+    __builtin_memmove(f4, f8, sizeof(float) * 4);
+
+// CHECK: call void @llvm.memset{{.*}}, !dbg [[G12R1:!.*]]
+    __builtin_memset(f4, 0, sizeof(float) * 4);
+
+// CHECK: call void @llvm.memset{{.*}}, !dbg [[G13R1:!.*]]
+    __builtin_memset_inline(f4, 0, sizeof(float) * 4);
+
+// CHECK: call void @llvm.memset{{.*}}, !dbg [[G14R1:!.*]]
+    __builtin___memset_chk(f4, 0, sizeof(float), -1);
+}
+
+// CHECK: [[G1R2]] = !DILocation({{.*}}, atomGroup: 1, atomRank: 2)
+// CHECK: [[G1R1]] = !DILocation({{.*}}, atomGroup: 1, atomRank: 1)
+// CHECK: [[G2R2]] = !DILocation({{.*}}, atomGroup: 2, atomRank: 2)
+// CHECK: [[G2R1]] = !DILocation({{.*}}, atomGroup: 2, atomRank: 1)
+// CHECK: [[G3R1]] = !DILocation({{.*}}, atomGroup: 3, atomRank: 1)
+// CHECK: [[G4R1]] = !DILocation({{.*}}, atomGroup: 4, atomRank: 1)
+// CHECK: [[G5R1]] = !DILocation({{.*}}, atomGroup: 5, atomRank: 1)
+// CHECK: [[G6R1]] = !DILocation({{.*}}, atomGroup: 6, atomRank: 1)
+// CHECK: [[G7R1]] = !DILocation({{.*}}, atomGroup: 7, atomRank: 1)
+// CHECK: [[G8R1]] = !DILocation({{.*}}, atomGroup: 8, atomRank: 1)
+// CHECK: [[G9R1]] = !DILocation({{.*}}, atomGroup: 9, atomRank: 1)
+// CHECK: [[G10R1]] = !DILocation({{.*}}, atomGroup: 10, atomRank: 1)
+// CHECK: [[G11R1]] = !DILocation({{.*}}, atomGroup: 11, atomRank: 1)
+// CHECK: [[G12R1]] = !DILocation({{.*}}, atomGroup: 12, atomRank: 1)
+// CHECK: [[G13R1]] = !DILocation({{.*}}, atomGroup: 13, atomRank: 1)
+// CHECK: [[G14R1]] = !DILocation({{.*}}, atomGroup: 14, atomRank: 1)