[FMV][AArch64] Allow user to override version priority. #146092

labrinea · 2025-06-27T14:57:30Z

Implements option one of the proposal ARM-software/acle#403

…ority.

llvmbot · 2025-06-27T14:58:05Z

@llvm/pr-subscribers-llvm-analysis
@llvm/pr-subscribers-clang-codegen
@llvm/pr-subscribers-backend-aarch64

@llvm/pr-subscribers-llvm-transforms

Author: Alexandros Lamprineas (labrinea)

Changes

Implements option one of the proposal ARM-software/acle#403

Patch is 24.34 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/146092.diff

12 Files Affected:

(modified) clang/lib/CodeGen/Targets/AArch64.cpp (+4-1)
(modified) clang/test/CodeGen/AArch64/fmv-duplicate-mangled-name.c (+16)
(added) clang/test/CodeGen/AArch64/fmv-explicit-priority.c (+193)
(modified) llvm/include/llvm/Analysis/TargetTransformInfo.h (+5-1)
(modified) llvm/include/llvm/Analysis/TargetTransformInfoImpl.h (+1)
(modified) llvm/include/llvm/TargetParser/AArch64FeatPriorities.inc (+6-1)
(modified) llvm/lib/Analysis/TargetTransformInfo.cpp (+4)
(modified) llvm/lib/Target/AArch64/AArch64FMV.td (+5)
(modified) llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp (+14-3)
(modified) llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h (+1)
(modified) llvm/lib/TargetParser/AArch64TargetParser.cpp (+19-10)
(modified) llvm/lib/Transforms/IPO/GlobalOpt.cpp (+15-7)

diff --git a/clang/lib/CodeGen/Targets/AArch64.cpp b/clang/lib/CodeGen/Targets/AArch64.cpp
index b82c46966cf0b..e2ede08942183 100644
--- a/clang/lib/CodeGen/Targets/AArch64.cpp
+++ b/clang/lib/CodeGen/Targets/AArch64.cpp
@@ -1336,10 +1336,13 @@ void AArch64ABIInfo::appendAttributeMangling(StringRef AttrStr,
   });
 
   llvm::SmallDenseSet<StringRef, 8> UniqueFeats;
-  for (auto &Feat : Features)
+  for (auto &Feat : Features) {
+    if (!getTarget().doesFeatureAffectCodeGen(Feat))
+      continue;
     if (auto Ext = llvm::AArch64::parseFMVExtension(Feat))
       if (UniqueFeats.insert(Ext->Name).second)
         Out << 'M' << Ext->Name;
+  }
 }
 
 std::unique_ptr<TargetCodeGenInfo>
diff --git a/clang/test/CodeGen/AArch64/fmv-duplicate-mangled-name.c b/clang/test/CodeGen/AArch64/fmv-duplicate-mangled-name.c
index e7e611e09542e..44a798a84a8a2 100644
--- a/clang/test/CodeGen/AArch64/fmv-duplicate-mangled-name.c
+++ b/clang/test/CodeGen/AArch64/fmv-duplicate-mangled-name.c
@@ -1,5 +1,7 @@
 // RUN: %clang_cc1 -triple aarch64-linux-gnu -verify -emit-llvm-only %s -DCHECK_IMPLICIT_DEFAULT
 // RUN: %clang_cc1 -triple aarch64-linux-gnu -verify -emit-llvm-only %s -DCHECK_EXPLICIT_DEFAULT
+// RUN: %clang_cc1 -triple aarch64-linux-gnu -verify -emit-llvm-only %s -DCHECK_EXPLICIT_VERSION_PRIORITY
+// RUN: %clang_cc1 -triple aarch64-linux-gnu -verify -emit-llvm-only %s -DCHECK_EXPLICIT_CLONES_PRIORITY
 
 #if defined(CHECK_IMPLICIT_DEFAULT)
 
@@ -21,4 +23,18 @@ __attribute__((target_version("default"))) int explicit_default_bad(void) { retu
 // expected-note@-2 {{previous definition is here}}
 __attribute__((target_clones("aes", "lse", "default"))) int explicit_default_bad(void) { return 1; }
 
+#elif defined(CHECK_EXPLICIT_VERSION_PRIORITY)
+
+__attribute__((target_version("aes"))) int explicit_version_priority(void) { return 0; }
+// expected-error@+2 {{definition with same mangled name 'explicit_version_priority._Maes' as another definition}}
+// expected-note@-2 {{previous definition is here}}
+__attribute__((target_version("priority1+aes"))) int explicit_version_priority(void) { return 1; }
+
+#elif defined(CHECK_EXPLICIT_CLONES_PRIORITY)
+
+__attribute__((target_version("aes+priority2"))) int explicit_clones_priority(void) { return 0; }
+// expected-error@+2 {{definition with same mangled name 'explicit_clones_priority._Maes' as another definition}}
+// expected-note@-2 {{previous definition is here}}
+__attribute__((target_clones("priority1+aes", "lse"))) int explicit_clones_priority(void) { return 1; }
+
 #endif
diff --git a/clang/test/CodeGen/AArch64/fmv-explicit-priority.c b/clang/test/CodeGen/AArch64/fmv-explicit-priority.c
new file mode 100644
index 0000000000000..95a246fea425b
--- /dev/null
+++ b/clang/test/CodeGen/AArch64/fmv-explicit-priority.c
@@ -0,0 +1,193 @@
+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py UTC_ARGS: --function-signature --check-attributes --check-globals --include-generated-funcs
+// RUN: %clang_cc1 -triple aarch64-none-linux-gnu -O3 -fno-inline -emit-llvm -o - %s | FileCheck %s
+
+__attribute__((target_version("priority1+lse"))) int foo(void) { return 1; }
+__attribute__((target_version("priority2+sve2"))) int foo(void) { return 2; }
+__attribute__((target_version("priority3+sve"))) int foo(void) { return 3; }
+__attribute__((target_version( "default"))) int foo(void) { return 0; }
+
+__attribute__((target_clones("priority1+lse+sve2", "priority2+lse", "priority3+sve", "default")))
+int fmv_caller(void) { return foo(); }
+
+
+__attribute__((target_version("aes"))) int bar(void) { return 1; }
+__attribute__((target_version("priority1+sm4"))) int bar(void) { return 2; }
+__attribute__((target_version("default"))) int bar(void) { return 0; }
+
+__attribute__((target("aes"))) int regular_caller_aes() { return bar(); }
+__attribute__((target("sm4"))) int regular_caller_sm4() { return bar(); }
+//.
+// CHECK: @__aarch64_cpu_features = external dso_local local_unnamed_addr global { i64 }
+// CHECK: @foo = weak_odr ifunc i32 (), ptr @foo.resolver
+// CHECK: @fmv_caller = weak_odr ifunc i32 (), ptr @fmv_caller.resolver
+// CHECK: @bar = weak_odr ifunc i32 (), ptr @bar.resolver
+//.
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none)
+// CHECK-LABEL: define {{[^@]+}}@foo._Mlse
+// CHECK-SAME: () #[[ATTR0:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    ret i32 1
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16)
+// CHECK-LABEL: define {{[^@]+}}@foo._Msve2
+// CHECK-SAME: () #[[ATTR1:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    ret i32 2
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16)
+// CHECK-LABEL: define {{[^@]+}}@foo._Msve
+// CHECK-SAME: () #[[ATTR2:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    ret i32 3
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none)
+// CHECK-LABEL: define {{[^@]+}}@foo.default
+// CHECK-SAME: () #[[ATTR3:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    ret i32 0
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16)
+// CHECK-LABEL: define {{[^@]+}}@fmv_caller._MlseMsve2
+// CHECK-SAME: () #[[ATTR4:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    [[CALL:%.*]] = tail call i32 @foo._Mlse()
+// CHECK-NEXT:    ret i32 [[CALL]]
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16)
+// CHECK-LABEL: define {{[^@]+}}@fmv_caller._Mlse
+// CHECK-SAME: () #[[ATTR5:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    [[CALL:%.*]] = tail call i32 @foo._Mlse()
+// CHECK-NEXT:    ret i32 [[CALL]]
+//
+//
+// CHECK: Function Attrs: noinline nounwind vscale_range(1,16)
+// CHECK-LABEL: define {{[^@]+}}@fmv_caller._Msve
+// CHECK-SAME: () #[[ATTR6:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    [[CALL:%.*]] = tail call i32 @foo() #[[ATTR12:[0-9]+]]
+// CHECK-NEXT:    ret i32 [[CALL]]
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16)
+// CHECK-LABEL: define {{[^@]+}}@fmv_caller.default
+// CHECK-SAME: () #[[ATTR7:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    [[CALL:%.*]] = tail call i32 @foo.default()
+// CHECK-NEXT:    ret i32 [[CALL]]
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none)
+// CHECK-LABEL: define {{[^@]+}}@bar._Maes
+// CHECK-SAME: () #[[ATTR8:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    ret i32 1
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none)
+// CHECK-LABEL: define {{[^@]+}}@bar._Msm4
+// CHECK-SAME: () #[[ATTR9:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    ret i32 2
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none)
+// CHECK-LABEL: define {{[^@]+}}@bar.default
+// CHECK-SAME: () #[[ATTR3]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    ret i32 0
+//
+//
+// CHECK: Function Attrs: noinline nounwind
+// CHECK-LABEL: define {{[^@]+}}@regular_caller_aes
+// CHECK-SAME: () local_unnamed_addr #[[ATTR10:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    [[CALL:%.*]] = tail call i32 @bar() #[[ATTR12]]
+// CHECK-NEXT:    ret i32 [[CALL]]
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none)
+// CHECK-LABEL: define {{[^@]+}}@regular_caller_sm4
+// CHECK-SAME: () local_unnamed_addr #[[ATTR11:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    [[CALL:%.*]] = tail call i32 @bar._Msm4()
+// CHECK-NEXT:    ret i32 [[CALL]]
+//
+//
+// CHECK-LABEL: define {{[^@]+}}@foo.resolver() comdat {
+// CHECK-NEXT:  resolver_entry:
+// CHECK-NEXT:    tail call void @__init_cpu_features_resolver()
+// CHECK-NEXT:    [[TMP0:%.*]] = load i64, ptr @__aarch64_cpu_features, align 8
+// CHECK-NEXT:    [[TMP1:%.*]] = and i64 [[TMP0]], 128
+// CHECK-NEXT:    [[DOTNOT:%.*]] = icmp eq i64 [[TMP1]], 0
+// CHECK-NEXT:    br i1 [[DOTNOT]], label [[RESOLVER_ELSE:%.*]], label [[COMMON_RET:%.*]]
+// CHECK:       common.ret:
+// CHECK-NEXT:    [[COMMON_RET_OP:%.*]] = phi ptr [ @foo._Mlse, [[RESOLVER_ENTRY:%.*]] ], [ @foo._Msve2, [[RESOLVER_ELSE]] ], [ [[FOO__MSVE_FOO_DEFAULT:%.*]], [[RESOLVER_ELSE2:%.*]] ]
+// CHECK-NEXT:    ret ptr [[COMMON_RET_OP]]
+// CHECK:       resolver_else:
+// CHECK-NEXT:    [[TMP2:%.*]] = and i64 [[TMP0]], 69793284352
+// CHECK-NEXT:    [[TMP3:%.*]] = icmp eq i64 [[TMP2]], 69793284352
+// CHECK-NEXT:    br i1 [[TMP3]], label [[COMMON_RET]], label [[RESOLVER_ELSE2]]
+// CHECK:       resolver_else2:
+// CHECK-NEXT:    [[TMP4:%.*]] = and i64 [[TMP0]], 1073807616
+// CHECK-NEXT:    [[TMP5:%.*]] = icmp eq i64 [[TMP4]], 1073807616
+// CHECK-NEXT:    [[FOO__MSVE_FOO_DEFAULT]] = select i1 [[TMP5]], ptr @foo._Msve, ptr @foo.default
+// CHECK-NEXT:    br label [[COMMON_RET]]
+//
+//
+// CHECK-LABEL: define {{[^@]+}}@fmv_caller.resolver() comdat {
+// CHECK-NEXT:  resolver_entry:
+// CHECK-NEXT:    tail call void @__init_cpu_features_resolver()
+// CHECK-NEXT:    [[TMP0:%.*]] = load i64, ptr @__aarch64_cpu_features, align 8
+// CHECK-NEXT:    [[TMP1:%.*]] = and i64 [[TMP0]], 69793284480
+// CHECK-NEXT:    [[TMP2:%.*]] = icmp eq i64 [[TMP1]], 69793284480
+// CHECK-NEXT:    br i1 [[TMP2]], label [[COMMON_RET:%.*]], label [[RESOLVER_ELSE:%.*]]
+// CHECK:       common.ret:
+// CHECK-NEXT:    [[COMMON_RET_OP:%.*]] = phi ptr [ @fmv_caller._MlseMsve2, [[RESOLVER_ENTRY:%.*]] ], [ @fmv_caller._Mlse, [[RESOLVER_ELSE]] ], [ [[FMV_CALLER__MSVE_FMV_CALLER_DEFAULT:%.*]], [[RESOLVER_ELSE2:%.*]] ]
+// CHECK-NEXT:    ret ptr [[COMMON_RET_OP]]
+// CHECK:       resolver_else:
+// CHECK-NEXT:    [[TMP3:%.*]] = and i64 [[TMP0]], 128
+// CHECK-NEXT:    [[DOTNOT:%.*]] = icmp eq i64 [[TMP3]], 0
+// CHECK-NEXT:    br i1 [[DOTNOT]], label [[RESOLVER_ELSE2]], label [[COMMON_RET]]
+// CHECK:       resolver_else2:
+// CHECK-NEXT:    [[TMP4:%.*]] = and i64 [[TMP0]], 1073807616
+// CHECK-NEXT:    [[TMP5:%.*]] = icmp eq i64 [[TMP4]], 1073807616
+// CHECK-NEXT:    [[FMV_CALLER__MSVE_FMV_CALLER_DEFAULT]] = select i1 [[TMP5]], ptr @fmv_caller._Msve, ptr @fmv_caller.default
+// CHECK-NEXT:    br label [[COMMON_RET]]
+//
+//
+// CHECK-LABEL: define {{[^@]+}}@bar.resolver() comdat {
+// CHECK-NEXT:  resolver_entry:
+// CHECK-NEXT:    tail call void @__init_cpu_features_resolver()
+// CHECK-NEXT:    [[TMP0:%.*]] = load i64, ptr @__aarch64_cpu_features, align 8
+// CHECK-NEXT:    [[TMP1:%.*]] = and i64 [[TMP0]], 800
+// CHECK-NEXT:    [[TMP2:%.*]] = icmp eq i64 [[TMP1]], 800
+// CHECK-NEXT:    [[TMP3:%.*]] = and i64 [[TMP0]], 33536
+// CHECK-NEXT:    [[TMP4:%.*]] = icmp eq i64 [[TMP3]], 33536
+// CHECK-NEXT:    [[BAR__MAES_BAR_DEFAULT:%.*]] = select i1 [[TMP4]], ptr @bar._Maes, ptr @bar.default
+// CHECK-NEXT:    [[COMMON_RET_OP:%.*]] = select i1 [[TMP2]], ptr @bar._Msm4, ptr [[BAR__MAES_BAR_DEFAULT]]
+// CHECK-NEXT:    ret ptr [[COMMON_RET_OP]]
+//
+//.
+// CHECK: attributes #[[ATTR0]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) "fmv-features"="lse,priority1" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+lse" }
+// CHECK: attributes #[[ATTR1]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16) "fmv-features"="priority2,sve2" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+fp-armv8,+fullfp16,+sve,+sve2" }
+// CHECK: attributes #[[ATTR2]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16) "fmv-features"="priority3,sve" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+fp-armv8,+fullfp16,+sve" }
+// CHECK: attributes #[[ATTR3]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) "fmv-features" "no-trapping-math"="true" "stack-protector-buffer-size"="8" }
+// CHECK: attributes #[[ATTR4]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16) "fmv-features"="lse,priority1,sve2" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+fp-armv8,+fullfp16,+lse,+sve,+sve2" }
+// CHECK: attributes #[[ATTR5]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16) "fmv-features"="lse,priority2" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+lse" }
+// CHECK: attributes #[[ATTR6]] = { noinline nounwind vscale_range(1,16) "fmv-features"="priority3,sve" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+fp-armv8,+fullfp16,+sve" }
+// CHECK: attributes #[[ATTR7]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16) "fmv-features" "no-trapping-math"="true" "stack-protector-buffer-size"="8" }
+// CHECK: attributes #[[ATTR8]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) "fmv-features"="aes" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+aes,+fp-armv8,+neon" }
+// CHECK: attributes #[[ATTR9]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) "fmv-features"="priority1,sm4" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+fp-armv8,+neon,+sm4" }
+// CHECK: attributes #[[ATTR10]] = { noinline nounwind "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+aes,+fp-armv8,+neon" }
+// CHECK: attributes #[[ATTR11]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+fp-armv8,+neon,+sm4" }
+// CHECK: attributes #[[ATTR12]] = { nounwind }
+//.
+// CHECK: [[META0:![0-9]+]] = !{i32 1, !"wchar_size", i32 4}
+// CHECK: [[META1:![0-9]+]] = !{!"{{.*}}clang version {{.*}}"}
+//.
diff --git a/llvm/include/llvm/Analysis/TargetTransformInfo.h b/llvm/include/llvm/Analysis/TargetTransformInfo.h
index 90d92e0fcf55c..c8e9a67b627f6 100644
--- a/llvm/include/llvm/Analysis/TargetTransformInfo.h
+++ b/llvm/include/llvm/Analysis/TargetTransformInfo.h
@@ -1916,9 +1916,13 @@ class TargetTransformInfo {
   LLVM_ABI bool hasArmWideBranch(bool Thumb) const;
 
   /// Returns a bitmask constructed from the target-features or fmv-features
-  /// metadata of a function.
+  /// metadata of a function corresponding to its Arch Extensions.
   LLVM_ABI uint64_t getFeatureMask(const Function &F) const;
 
+  /// Returns a bitmask constructed from the target-features or fmv-features
+  /// metadata of a function corresponding to its FMV priority.
+  LLVM_ABI uint64_t getPriorityMask(const Function &F) const;
+
   /// Returns true if this is an instance of a function with multiple versions.
   LLVM_ABI bool isMultiversionedFunction(const Function &F) const;
 
diff --git a/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h b/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
index c22928c9bcd94..628c92ddebd21 100644
--- a/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
+++ b/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
@@ -1126,6 +1126,7 @@ class TargetTransformInfoImplBase {
   virtual bool hasArmWideBranch(bool) const { return false; }
 
   virtual uint64_t getFeatureMask(const Function &F) const { return 0; }
+  virtual uint64_t getPriorityMask(const Function &F) const { return 0; }
 
   virtual bool isMultiversionedFunction(const Function &F) const {
     return false;
diff --git a/llvm/include/llvm/TargetParser/AArch64FeatPriorities.inc b/llvm/include/llvm/TargetParser/AArch64FeatPriorities.inc
index f2bad28ada93e..cde0d16e0b32e 100644
--- a/llvm/include/llvm/TargetParser/AArch64FeatPriorities.inc
+++ b/llvm/include/llvm/TargetParser/AArch64FeatPriorities.inc
@@ -59,7 +59,12 @@ enum FeatPriorities {
   PRIOR_SME_I64,
   PRIOR_SME2,
   PRIOR_MOPS,
-  PRIOR_CSSC
+  PRIOR_CSSC,
+  PRIOR_5,
+  PRIOR_4,
+  PRIOR_3,
+  PRIOR_2,
+  PRIOR_1
 };
 
 #endif
diff --git a/llvm/lib/Analysis/TargetTransformInfo.cpp b/llvm/lib/Analysis/TargetTransformInfo.cpp
index 3ebd9d487ba04..fb4002e277616 100644
--- a/llvm/lib/Analysis/TargetTransformInfo.cpp
+++ b/llvm/lib/Analysis/TargetTransformInfo.cpp
@@ -1426,6 +1426,10 @@ uint64_t TargetTransformInfo::getFeatureMask(const Function &F) const {
   return TTIImpl->getFeatureMask(F);
 }
 
+uint64_t TargetTransformInfo::getPriorityMask(const Function &F) const {
+  return TTIImpl->getPriorityMask(F);
+}
+
 bool TargetTransformInfo::isMultiversionedFunction(const Function &F) const {
   return TTIImpl->isMultiversionedFunction(F);
 }
diff --git a/llvm/lib/Target/AArch64/AArch64FMV.td b/llvm/lib/Target/AArch64/AArch64FMV.td
index b0f76ec6a6480..efcb6f552d788 100644
--- a/llvm/lib/Target/AArch64/AArch64FMV.td
+++ b/llvm/lib/Target/AArch64/AArch64FMV.td
@@ -83,3 +83,8 @@ def : FMVExtension<"sve2-sha3", "SVE_SHA3">;
 def : FMVExtension<"sve2-sm4", "SVE_SM4">;
 def : FMVExtension<"wfxt", "WFXT">;
 def : FMVExtension<"cssc", "CSSC">;
+let FeatureBit = "FEAT_MAX" in def : FMVExtension<"priority1", "1">;
+let FeatureBit = "FEAT_MAX" in def : FMVExtension<"priority2", "2">;
+let FeatureBit = "FEAT_MAX" in def : FMVExtension<"priority3", "3">;
+let FeatureBit = "FEAT_MAX" in def : FMVExtension<"priority4", "4">;
+let FeatureBit = "FEAT_MAX" in def : FMVExtension<"priority5", "5">;
diff --git a/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp b/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
index 3387dee8aa4c8..f920efc0386ea 100644
--- a/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
+++ b/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
@@ -249,12 +249,23 @@ static bool hasPossibleIncompatibleOps(const Function *F) {
   return false;
 }
 
-uint64_t AArch64TTIImpl::getFeatureMask(const Function &F) const {
+static void extractAttrFeatures(const Function &F, const AArch64TTIImpl *TTI,
+                                SmallVectorImpl<StringRef> &Features) {
   StringRef AttributeStr =
-      isMultiversionedFunction(F) ? "fmv-features" : "target-features";
+      TTI->isMultiversionedFunction(F) ? "fmv-features" : "target-features";
   StringRef FeatureStr = F.getFnAttribute(AttributeStr).getValueAsString();
-  SmallVector<StringRef, 8> Features;
   FeatureStr.split(Features, ",");
+}
+
+uint64_t AArch64TTIImpl::getFeatureMask(const Function &F) const {
+  SmallVector<StringRef, 8> Features;
+  extractAttrFeatures(F, this, Features);
+  return AArch64::getCpuSupportsMask(Features);
+}
+
+uint64_t AArch64TTIImpl::getPriorityMask(const Function &F) const {
+  SmallVector<StringRef, 8> Features;
+  extractAttrFeatures(F, this, Features);
   return AArch64::getFMVPriority(Features);
 }
 
diff --git a/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h b/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h
index 9ada70bd7086a..c653e06785eee 100644
--- a/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h
+++ b/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h
@@ -92,6 +92,7 @@ class AArch64TTIImpl final : public BasicTTIImplBase<AArch64TTIImpl> {
                                 unsigned DefaultCallPenalty) const override;
 
   uint64_t getFeatureMask(const Function &F) const override;
+  uint64_t getPriorityMask(const Function &F) const override;
 
   bool isMultiversionedFunction(const Function &F) const override;
 
diff --git a/llvm/lib/TargetParser/AArch64TargetParser.cpp b/llvm/lib/TargetParser/AArch64TargetParser.cpp
index 4a2523440f0f0..37d82abf34db8 100644
--- a/llvm/lib/TargetParser/AArch64TargetParser.cpp
+++ b/llvm/lib/TargetParser/AArch64TargetParser.cpp
@@ -55,21 +55,30 @@ std::optional<AArch64::FMVInfo> lookupFMVByID(AArch64::ArchExtKind ExtID) {
   return {};
 }
 
+std::optional<AArch64::FMVInfo> getFMVInfoFrom(StringRef Feature) {
+  std::optional<AArch64::FMVInfo> FMV = AArch64::parseFMVExtension(Feature);
+  if (!FMV && Feature.starts_with('+'))
+    if (std::optional<AArch64::ExtensionInfo> Ext =
+            AArch64::targetFeatureToExtension(Feature))
+      FMV = lookupFMVByID(Ext->ID);
+ ...
[truncated]

llvmbot · 2025-06-27T14:58:07Z

@llvm/pr-subscribers-clang

Author: Alexandros Lamprineas (labrinea)

Changes

Implements option one of the proposal ARM-software/acle#403

Patch is 24.34 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/146092.diff

12 Files Affected:

(modified) clang/lib/CodeGen/Targets/AArch64.cpp (+4-1)
(modified) clang/test/CodeGen/AArch64/fmv-duplicate-mangled-name.c (+16)
(added) clang/test/CodeGen/AArch64/fmv-explicit-priority.c (+193)
(modified) llvm/include/llvm/Analysis/TargetTransformInfo.h (+5-1)
(modified) llvm/include/llvm/Analysis/TargetTransformInfoImpl.h (+1)
(modified) llvm/include/llvm/TargetParser/AArch64FeatPriorities.inc (+6-1)
(modified) llvm/lib/Analysis/TargetTransformInfo.cpp (+4)
(modified) llvm/lib/Target/AArch64/AArch64FMV.td (+5)
(modified) llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp (+14-3)
(modified) llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h (+1)
(modified) llvm/lib/TargetParser/AArch64TargetParser.cpp (+19-10)
(modified) llvm/lib/Transforms/IPO/GlobalOpt.cpp (+15-7)

diff --git a/clang/lib/CodeGen/Targets/AArch64.cpp b/clang/lib/CodeGen/Targets/AArch64.cpp
index b82c46966cf0b..e2ede08942183 100644
--- a/clang/lib/CodeGen/Targets/AArch64.cpp
+++ b/clang/lib/CodeGen/Targets/AArch64.cpp
@@ -1336,10 +1336,13 @@ void AArch64ABIInfo::appendAttributeMangling(StringRef AttrStr,
   });
 
   llvm::SmallDenseSet<StringRef, 8> UniqueFeats;
-  for (auto &Feat : Features)
+  for (auto &Feat : Features) {
+    if (!getTarget().doesFeatureAffectCodeGen(Feat))
+      continue;
     if (auto Ext = llvm::AArch64::parseFMVExtension(Feat))
       if (UniqueFeats.insert(Ext->Name).second)
         Out << 'M' << Ext->Name;
+  }
 }
 
 std::unique_ptr<TargetCodeGenInfo>
diff --git a/clang/test/CodeGen/AArch64/fmv-duplicate-mangled-name.c b/clang/test/CodeGen/AArch64/fmv-duplicate-mangled-name.c
index e7e611e09542e..44a798a84a8a2 100644
--- a/clang/test/CodeGen/AArch64/fmv-duplicate-mangled-name.c
+++ b/clang/test/CodeGen/AArch64/fmv-duplicate-mangled-name.c
@@ -1,5 +1,7 @@
 // RUN: %clang_cc1 -triple aarch64-linux-gnu -verify -emit-llvm-only %s -DCHECK_IMPLICIT_DEFAULT
 // RUN: %clang_cc1 -triple aarch64-linux-gnu -verify -emit-llvm-only %s -DCHECK_EXPLICIT_DEFAULT
+// RUN: %clang_cc1 -triple aarch64-linux-gnu -verify -emit-llvm-only %s -DCHECK_EXPLICIT_VERSION_PRIORITY
+// RUN: %clang_cc1 -triple aarch64-linux-gnu -verify -emit-llvm-only %s -DCHECK_EXPLICIT_CLONES_PRIORITY
 
 #if defined(CHECK_IMPLICIT_DEFAULT)
 
@@ -21,4 +23,18 @@ __attribute__((target_version("default"))) int explicit_default_bad(void) { retu
 // expected-note@-2 {{previous definition is here}}
 __attribute__((target_clones("aes", "lse", "default"))) int explicit_default_bad(void) { return 1; }
 
+#elif defined(CHECK_EXPLICIT_VERSION_PRIORITY)
+
+__attribute__((target_version("aes"))) int explicit_version_priority(void) { return 0; }
+// expected-error@+2 {{definition with same mangled name 'explicit_version_priority._Maes' as another definition}}
+// expected-note@-2 {{previous definition is here}}
+__attribute__((target_version("priority1+aes"))) int explicit_version_priority(void) { return 1; }
+
+#elif defined(CHECK_EXPLICIT_CLONES_PRIORITY)
+
+__attribute__((target_version("aes+priority2"))) int explicit_clones_priority(void) { return 0; }
+// expected-error@+2 {{definition with same mangled name 'explicit_clones_priority._Maes' as another definition}}
+// expected-note@-2 {{previous definition is here}}
+__attribute__((target_clones("priority1+aes", "lse"))) int explicit_clones_priority(void) { return 1; }
+
 #endif
diff --git a/clang/test/CodeGen/AArch64/fmv-explicit-priority.c b/clang/test/CodeGen/AArch64/fmv-explicit-priority.c
new file mode 100644
index 0000000000000..95a246fea425b
--- /dev/null
+++ b/clang/test/CodeGen/AArch64/fmv-explicit-priority.c
@@ -0,0 +1,193 @@
+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py UTC_ARGS: --function-signature --check-attributes --check-globals --include-generated-funcs
+// RUN: %clang_cc1 -triple aarch64-none-linux-gnu -O3 -fno-inline -emit-llvm -o - %s | FileCheck %s
+
+__attribute__((target_version("priority1+lse"))) int foo(void) { return 1; }
+__attribute__((target_version("priority2+sve2"))) int foo(void) { return 2; }
+__attribute__((target_version("priority3+sve"))) int foo(void) { return 3; }
+__attribute__((target_version( "default"))) int foo(void) { return 0; }
+
+__attribute__((target_clones("priority1+lse+sve2", "priority2+lse", "priority3+sve", "default")))
+int fmv_caller(void) { return foo(); }
+
+
+__attribute__((target_version("aes"))) int bar(void) { return 1; }
+__attribute__((target_version("priority1+sm4"))) int bar(void) { return 2; }
+__attribute__((target_version("default"))) int bar(void) { return 0; }
+
+__attribute__((target("aes"))) int regular_caller_aes() { return bar(); }
+__attribute__((target("sm4"))) int regular_caller_sm4() { return bar(); }
+//.
+// CHECK: @__aarch64_cpu_features = external dso_local local_unnamed_addr global { i64 }
+// CHECK: @foo = weak_odr ifunc i32 (), ptr @foo.resolver
+// CHECK: @fmv_caller = weak_odr ifunc i32 (), ptr @fmv_caller.resolver
+// CHECK: @bar = weak_odr ifunc i32 (), ptr @bar.resolver
+//.
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none)
+// CHECK-LABEL: define {{[^@]+}}@foo._Mlse
+// CHECK-SAME: () #[[ATTR0:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    ret i32 1
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16)
+// CHECK-LABEL: define {{[^@]+}}@foo._Msve2
+// CHECK-SAME: () #[[ATTR1:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    ret i32 2
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16)
+// CHECK-LABEL: define {{[^@]+}}@foo._Msve
+// CHECK-SAME: () #[[ATTR2:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    ret i32 3
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none)
+// CHECK-LABEL: define {{[^@]+}}@foo.default
+// CHECK-SAME: () #[[ATTR3:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    ret i32 0
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16)
+// CHECK-LABEL: define {{[^@]+}}@fmv_caller._MlseMsve2
+// CHECK-SAME: () #[[ATTR4:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    [[CALL:%.*]] = tail call i32 @foo._Mlse()
+// CHECK-NEXT:    ret i32 [[CALL]]
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16)
+// CHECK-LABEL: define {{[^@]+}}@fmv_caller._Mlse
+// CHECK-SAME: () #[[ATTR5:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    [[CALL:%.*]] = tail call i32 @foo._Mlse()
+// CHECK-NEXT:    ret i32 [[CALL]]
+//
+//
+// CHECK: Function Attrs: noinline nounwind vscale_range(1,16)
+// CHECK-LABEL: define {{[^@]+}}@fmv_caller._Msve
+// CHECK-SAME: () #[[ATTR6:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    [[CALL:%.*]] = tail call i32 @foo() #[[ATTR12:[0-9]+]]
+// CHECK-NEXT:    ret i32 [[CALL]]
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16)
+// CHECK-LABEL: define {{[^@]+}}@fmv_caller.default
+// CHECK-SAME: () #[[ATTR7:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    [[CALL:%.*]] = tail call i32 @foo.default()
+// CHECK-NEXT:    ret i32 [[CALL]]
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none)
+// CHECK-LABEL: define {{[^@]+}}@bar._Maes
+// CHECK-SAME: () #[[ATTR8:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    ret i32 1
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none)
+// CHECK-LABEL: define {{[^@]+}}@bar._Msm4
+// CHECK-SAME: () #[[ATTR9:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    ret i32 2
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none)
+// CHECK-LABEL: define {{[^@]+}}@bar.default
+// CHECK-SAME: () #[[ATTR3]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    ret i32 0
+//
+//
+// CHECK: Function Attrs: noinline nounwind
+// CHECK-LABEL: define {{[^@]+}}@regular_caller_aes
+// CHECK-SAME: () local_unnamed_addr #[[ATTR10:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    [[CALL:%.*]] = tail call i32 @bar() #[[ATTR12]]
+// CHECK-NEXT:    ret i32 [[CALL]]
+//
+//
+// CHECK: Function Attrs: mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none)
+// CHECK-LABEL: define {{[^@]+}}@regular_caller_sm4
+// CHECK-SAME: () local_unnamed_addr #[[ATTR11:[0-9]+]] {
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    [[CALL:%.*]] = tail call i32 @bar._Msm4()
+// CHECK-NEXT:    ret i32 [[CALL]]
+//
+//
+// CHECK-LABEL: define {{[^@]+}}@foo.resolver() comdat {
+// CHECK-NEXT:  resolver_entry:
+// CHECK-NEXT:    tail call void @__init_cpu_features_resolver()
+// CHECK-NEXT:    [[TMP0:%.*]] = load i64, ptr @__aarch64_cpu_features, align 8
+// CHECK-NEXT:    [[TMP1:%.*]] = and i64 [[TMP0]], 128
+// CHECK-NEXT:    [[DOTNOT:%.*]] = icmp eq i64 [[TMP1]], 0
+// CHECK-NEXT:    br i1 [[DOTNOT]], label [[RESOLVER_ELSE:%.*]], label [[COMMON_RET:%.*]]
+// CHECK:       common.ret:
+// CHECK-NEXT:    [[COMMON_RET_OP:%.*]] = phi ptr [ @foo._Mlse, [[RESOLVER_ENTRY:%.*]] ], [ @foo._Msve2, [[RESOLVER_ELSE]] ], [ [[FOO__MSVE_FOO_DEFAULT:%.*]], [[RESOLVER_ELSE2:%.*]] ]
+// CHECK-NEXT:    ret ptr [[COMMON_RET_OP]]
+// CHECK:       resolver_else:
+// CHECK-NEXT:    [[TMP2:%.*]] = and i64 [[TMP0]], 69793284352
+// CHECK-NEXT:    [[TMP3:%.*]] = icmp eq i64 [[TMP2]], 69793284352
+// CHECK-NEXT:    br i1 [[TMP3]], label [[COMMON_RET]], label [[RESOLVER_ELSE2]]
+// CHECK:       resolver_else2:
+// CHECK-NEXT:    [[TMP4:%.*]] = and i64 [[TMP0]], 1073807616
+// CHECK-NEXT:    [[TMP5:%.*]] = icmp eq i64 [[TMP4]], 1073807616
+// CHECK-NEXT:    [[FOO__MSVE_FOO_DEFAULT]] = select i1 [[TMP5]], ptr @foo._Msve, ptr @foo.default
+// CHECK-NEXT:    br label [[COMMON_RET]]
+//
+//
+// CHECK-LABEL: define {{[^@]+}}@fmv_caller.resolver() comdat {
+// CHECK-NEXT:  resolver_entry:
+// CHECK-NEXT:    tail call void @__init_cpu_features_resolver()
+// CHECK-NEXT:    [[TMP0:%.*]] = load i64, ptr @__aarch64_cpu_features, align 8
+// CHECK-NEXT:    [[TMP1:%.*]] = and i64 [[TMP0]], 69793284480
+// CHECK-NEXT:    [[TMP2:%.*]] = icmp eq i64 [[TMP1]], 69793284480
+// CHECK-NEXT:    br i1 [[TMP2]], label [[COMMON_RET:%.*]], label [[RESOLVER_ELSE:%.*]]
+// CHECK:       common.ret:
+// CHECK-NEXT:    [[COMMON_RET_OP:%.*]] = phi ptr [ @fmv_caller._MlseMsve2, [[RESOLVER_ENTRY:%.*]] ], [ @fmv_caller._Mlse, [[RESOLVER_ELSE]] ], [ [[FMV_CALLER__MSVE_FMV_CALLER_DEFAULT:%.*]], [[RESOLVER_ELSE2:%.*]] ]
+// CHECK-NEXT:    ret ptr [[COMMON_RET_OP]]
+// CHECK:       resolver_else:
+// CHECK-NEXT:    [[TMP3:%.*]] = and i64 [[TMP0]], 128
+// CHECK-NEXT:    [[DOTNOT:%.*]] = icmp eq i64 [[TMP3]], 0
+// CHECK-NEXT:    br i1 [[DOTNOT]], label [[RESOLVER_ELSE2]], label [[COMMON_RET]]
+// CHECK:       resolver_else2:
+// CHECK-NEXT:    [[TMP4:%.*]] = and i64 [[TMP0]], 1073807616
+// CHECK-NEXT:    [[TMP5:%.*]] = icmp eq i64 [[TMP4]], 1073807616
+// CHECK-NEXT:    [[FMV_CALLER__MSVE_FMV_CALLER_DEFAULT]] = select i1 [[TMP5]], ptr @fmv_caller._Msve, ptr @fmv_caller.default
+// CHECK-NEXT:    br label [[COMMON_RET]]
+//
+//
+// CHECK-LABEL: define {{[^@]+}}@bar.resolver() comdat {
+// CHECK-NEXT:  resolver_entry:
+// CHECK-NEXT:    tail call void @__init_cpu_features_resolver()
+// CHECK-NEXT:    [[TMP0:%.*]] = load i64, ptr @__aarch64_cpu_features, align 8
+// CHECK-NEXT:    [[TMP1:%.*]] = and i64 [[TMP0]], 800
+// CHECK-NEXT:    [[TMP2:%.*]] = icmp eq i64 [[TMP1]], 800
+// CHECK-NEXT:    [[TMP3:%.*]] = and i64 [[TMP0]], 33536
+// CHECK-NEXT:    [[TMP4:%.*]] = icmp eq i64 [[TMP3]], 33536
+// CHECK-NEXT:    [[BAR__MAES_BAR_DEFAULT:%.*]] = select i1 [[TMP4]], ptr @bar._Maes, ptr @bar.default
+// CHECK-NEXT:    [[COMMON_RET_OP:%.*]] = select i1 [[TMP2]], ptr @bar._Msm4, ptr [[BAR__MAES_BAR_DEFAULT]]
+// CHECK-NEXT:    ret ptr [[COMMON_RET_OP]]
+//
+//.
+// CHECK: attributes #[[ATTR0]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) "fmv-features"="lse,priority1" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+lse" }
+// CHECK: attributes #[[ATTR1]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16) "fmv-features"="priority2,sve2" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+fp-armv8,+fullfp16,+sve,+sve2" }
+// CHECK: attributes #[[ATTR2]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16) "fmv-features"="priority3,sve" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+fp-armv8,+fullfp16,+sve" }
+// CHECK: attributes #[[ATTR3]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) "fmv-features" "no-trapping-math"="true" "stack-protector-buffer-size"="8" }
+// CHECK: attributes #[[ATTR4]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16) "fmv-features"="lse,priority1,sve2" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+fp-armv8,+fullfp16,+lse,+sve,+sve2" }
+// CHECK: attributes #[[ATTR5]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16) "fmv-features"="lse,priority2" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+lse" }
+// CHECK: attributes #[[ATTR6]] = { noinline nounwind vscale_range(1,16) "fmv-features"="priority3,sve" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+fp-armv8,+fullfp16,+sve" }
+// CHECK: attributes #[[ATTR7]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) vscale_range(1,16) "fmv-features" "no-trapping-math"="true" "stack-protector-buffer-size"="8" }
+// CHECK: attributes #[[ATTR8]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) "fmv-features"="aes" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+aes,+fp-armv8,+neon" }
+// CHECK: attributes #[[ATTR9]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) "fmv-features"="priority1,sm4" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+fp-armv8,+neon,+sm4" }
+// CHECK: attributes #[[ATTR10]] = { noinline nounwind "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+aes,+fp-armv8,+neon" }
+// CHECK: attributes #[[ATTR11]] = { mustprogress nofree noinline norecurse nosync nounwind willreturn memory(none) "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-features"="+fp-armv8,+neon,+sm4" }
+// CHECK: attributes #[[ATTR12]] = { nounwind }
+//.
+// CHECK: [[META0:![0-9]+]] = !{i32 1, !"wchar_size", i32 4}
+// CHECK: [[META1:![0-9]+]] = !{!"{{.*}}clang version {{.*}}"}
+//.
diff --git a/llvm/include/llvm/Analysis/TargetTransformInfo.h b/llvm/include/llvm/Analysis/TargetTransformInfo.h
index 90d92e0fcf55c..c8e9a67b627f6 100644
--- a/llvm/include/llvm/Analysis/TargetTransformInfo.h
+++ b/llvm/include/llvm/Analysis/TargetTransformInfo.h
@@ -1916,9 +1916,13 @@ class TargetTransformInfo {
   LLVM_ABI bool hasArmWideBranch(bool Thumb) const;
 
   /// Returns a bitmask constructed from the target-features or fmv-features
-  /// metadata of a function.
+  /// metadata of a function corresponding to its Arch Extensions.
   LLVM_ABI uint64_t getFeatureMask(const Function &F) const;
 
+  /// Returns a bitmask constructed from the target-features or fmv-features
+  /// metadata of a function corresponding to its FMV priority.
+  LLVM_ABI uint64_t getPriorityMask(const Function &F) const;
+
   /// Returns true if this is an instance of a function with multiple versions.
   LLVM_ABI bool isMultiversionedFunction(const Function &F) const;
 
diff --git a/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h b/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
index c22928c9bcd94..628c92ddebd21 100644
--- a/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
+++ b/llvm/include/llvm/Analysis/TargetTransformInfoImpl.h
@@ -1126,6 +1126,7 @@ class TargetTransformInfoImplBase {
   virtual bool hasArmWideBranch(bool) const { return false; }
 
   virtual uint64_t getFeatureMask(const Function &F) const { return 0; }
+  virtual uint64_t getPriorityMask(const Function &F) const { return 0; }
 
   virtual bool isMultiversionedFunction(const Function &F) const {
     return false;
diff --git a/llvm/include/llvm/TargetParser/AArch64FeatPriorities.inc b/llvm/include/llvm/TargetParser/AArch64FeatPriorities.inc
index f2bad28ada93e..cde0d16e0b32e 100644
--- a/llvm/include/llvm/TargetParser/AArch64FeatPriorities.inc
+++ b/llvm/include/llvm/TargetParser/AArch64FeatPriorities.inc
@@ -59,7 +59,12 @@ enum FeatPriorities {
   PRIOR_SME_I64,
   PRIOR_SME2,
   PRIOR_MOPS,
-  PRIOR_CSSC
+  PRIOR_CSSC,
+  PRIOR_5,
+  PRIOR_4,
+  PRIOR_3,
+  PRIOR_2,
+  PRIOR_1
 };
 
 #endif
diff --git a/llvm/lib/Analysis/TargetTransformInfo.cpp b/llvm/lib/Analysis/TargetTransformInfo.cpp
index 3ebd9d487ba04..fb4002e277616 100644
--- a/llvm/lib/Analysis/TargetTransformInfo.cpp
+++ b/llvm/lib/Analysis/TargetTransformInfo.cpp
@@ -1426,6 +1426,10 @@ uint64_t TargetTransformInfo::getFeatureMask(const Function &F) const {
   return TTIImpl->getFeatureMask(F);
 }
 
+uint64_t TargetTransformInfo::getPriorityMask(const Function &F) const {
+  return TTIImpl->getPriorityMask(F);
+}
+
 bool TargetTransformInfo::isMultiversionedFunction(const Function &F) const {
   return TTIImpl->isMultiversionedFunction(F);
 }
diff --git a/llvm/lib/Target/AArch64/AArch64FMV.td b/llvm/lib/Target/AArch64/AArch64FMV.td
index b0f76ec6a6480..efcb6f552d788 100644
--- a/llvm/lib/Target/AArch64/AArch64FMV.td
+++ b/llvm/lib/Target/AArch64/AArch64FMV.td
@@ -83,3 +83,8 @@ def : FMVExtension<"sve2-sha3", "SVE_SHA3">;
 def : FMVExtension<"sve2-sm4", "SVE_SM4">;
 def : FMVExtension<"wfxt", "WFXT">;
 def : FMVExtension<"cssc", "CSSC">;
+let FeatureBit = "FEAT_MAX" in def : FMVExtension<"priority1", "1">;
+let FeatureBit = "FEAT_MAX" in def : FMVExtension<"priority2", "2">;
+let FeatureBit = "FEAT_MAX" in def : FMVExtension<"priority3", "3">;
+let FeatureBit = "FEAT_MAX" in def : FMVExtension<"priority4", "4">;
+let FeatureBit = "FEAT_MAX" in def : FMVExtension<"priority5", "5">;
diff --git a/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp b/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
index 3387dee8aa4c8..f920efc0386ea 100644
--- a/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
+++ b/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
@@ -249,12 +249,23 @@ static bool hasPossibleIncompatibleOps(const Function *F) {
   return false;
 }
 
-uint64_t AArch64TTIImpl::getFeatureMask(const Function &F) const {
+static void extractAttrFeatures(const Function &F, const AArch64TTIImpl *TTI,
+                                SmallVectorImpl<StringRef> &Features) {
   StringRef AttributeStr =
-      isMultiversionedFunction(F) ? "fmv-features" : "target-features";
+      TTI->isMultiversionedFunction(F) ? "fmv-features" : "target-features";
   StringRef FeatureStr = F.getFnAttribute(AttributeStr).getValueAsString();
-  SmallVector<StringRef, 8> Features;
   FeatureStr.split(Features, ",");
+}
+
+uint64_t AArch64TTIImpl::getFeatureMask(const Function &F) const {
+  SmallVector<StringRef, 8> Features;
+  extractAttrFeatures(F, this, Features);
+  return AArch64::getCpuSupportsMask(Features);
+}
+
+uint64_t AArch64TTIImpl::getPriorityMask(const Function &F) const {
+  SmallVector<StringRef, 8> Features;
+  extractAttrFeatures(F, this, Features);
   return AArch64::getFMVPriority(Features);
 }
 
diff --git a/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h b/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h
index 9ada70bd7086a..c653e06785eee 100644
--- a/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h
+++ b/llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h
@@ -92,6 +92,7 @@ class AArch64TTIImpl final : public BasicTTIImplBase<AArch64TTIImpl> {
                                 unsigned DefaultCallPenalty) const override;
 
   uint64_t getFeatureMask(const Function &F) const override;
+  uint64_t getPriorityMask(const Function &F) const override;
 
   bool isMultiversionedFunction(const Function &F) const override;
 
diff --git a/llvm/lib/TargetParser/AArch64TargetParser.cpp b/llvm/lib/TargetParser/AArch64TargetParser.cpp
index 4a2523440f0f0..37d82abf34db8 100644
--- a/llvm/lib/TargetParser/AArch64TargetParser.cpp
+++ b/llvm/lib/TargetParser/AArch64TargetParser.cpp
@@ -55,21 +55,30 @@ std::optional<AArch64::FMVInfo> lookupFMVByID(AArch64::ArchExtKind ExtID) {
   return {};
 }
 
+std::optional<AArch64::FMVInfo> getFMVInfoFrom(StringRef Feature) {
+  std::optional<AArch64::FMVInfo> FMV = AArch64::parseFMVExtension(Feature);
+  if (!FMV && Feature.starts_with('+'))
+    if (std::optional<AArch64::ExtensionInfo> Ext =
+            AArch64::targetFeatureToExtension(Feature))
+      FMV = lookupFMVByID(Ext->ID);
+ ...
[truncated]

labrinea added 2 commits June 27, 2025 15:39

Supports option 1 of ARM-software/acle#403

c5bd398

Adds support for static resolution of calls with explicit version pri…

c5e6d92

…ority.

labrinea marked this pull request as draft June 27, 2025 14:57

llvmbot added clang Clang issues not falling into any other category backend:AArch64 clang:codegen IR generation bugs: mangling, exceptions, etc. llvm:analysis Includes value tracking, cost tables and constant folding llvm:transforms labels Jun 27, 2025

labrinea mentioned this pull request Jun 27, 2025

[proposal] Add syntax for expressing priority of FMV target expressions ARM-software/acle#403

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[FMV][AArch64] Allow user to override version priority. #146092

[FMV][AArch64] Allow user to override version priority. #146092

Uh oh!

labrinea commented Jun 27, 2025

Uh oh!

llvmbot commented Jun 27, 2025 •

edited

Loading

Uh oh!

llvmbot commented Jun 27, 2025

Uh oh!

Uh oh!

[FMV][AArch64] Allow user to override version priority. #146092

Are you sure you want to change the base?

[FMV][AArch64] Allow user to override version priority. #146092

Uh oh!

Conversation

labrinea commented Jun 27, 2025

Uh oh!

llvmbot commented Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

llvmbot commented Jun 27, 2025

Uh oh!

Uh oh!

llvmbot commented Jun 27, 2025 •

edited

Loading