Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[LLVM][AArch64] Add assembly/disassembly for SME2p2 ftmopa and bftmopa #112876

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

SpencerAbson
Copy link
Contributor

@SpencerAbson SpencerAbson commented Oct 18, 2024

note This work is based off of another change which adds the register classes and operands seen here, it is currently in review: #111717. This patch has been created early to provide verification for the implementation of the ZK register class, which the above work could not do in isolation. Thanks.

This patch adds assembly/disassembly for the following SME2p2 instructions (part of the 2024 AArch64 ISA update)

  • BFTMOPA (widening) - FEAT_SME2p2
  • BFTMOPA (non-widening) - FEAT_SME2p2 & FEAT_SME_B16B16
  • FTMOPA (4-way) - FEAT_SME2p2 & FEAT_SME_F8F32
  • FTMOPA (2-way, 8-to-16) - FEAT_SME2p2 & FEAT_SME_F8F16
  • FTMOPA (2-way, 16-to-32) - FEAT_SME2p2
  • FTMOPA (non-widening, f16) - FEAT_SME2p2 & FEAT_SME_F16F16
  • FTMOPA (non-widening, f32) - FEAT_SME2p2
    In accordance with: https://developer.arm.com/documentation/ddi0602/latest/

It also introduces .arch assembler tests for the new sme2p2 feature.
Co-authored-by: Marian Lukac marian.lukac@arm.com

Add new register classes/operands and their encoder/decoder behaviour required for the new Armv9.6
instructions (see https://developer.arm.com/documentation/109697/2024_09/Feature-descriptions/The-Armv9-6-architecture-extension).

This work is the basis ofthe 2024 Armv9.6 architecture update effort for SME.

Co-authored-by: Caroline Concatto caroline.concatto@arm.com
Co-authored-by: Marian Lukac marian.lukac@arm.com
Co-authored-by: Momchil Velikov momchil.velikov@arm.com
This patch adds assembly/disassembly for the following SME2p2 instructions (part of the 2024 AArch64 ISA update)
	- BFTMOPA (widening)         - FEAT_SME2p2
	- BFTMOPA (non-widening)     - FEAT_SME2p2 & FEAT_SME_B16B16
	- FTMOPA (4-way)             - FEAT_SME2p2 & FEAT_SME_F8F32
	- FTMOPA (2-way, 8-to-16)    - FEAT_SME2p2 & FEAT_SME_F8F16
	- FTMOPA (2-way, 16-to-32)   - FEAT_SME2p2
	- FTMOPA (non-widening, f16) - FEAT_SME2p2 & FEAT_SME_F16F16
	- FTMOPA (non-widening, f32) - FEAT_SME2p2

- It also introduces .arch assembler tests for the new sme2p2 feature
In accordance with: https://developer.arm.com/documentation/ddi0602/latest/

Co-authored-by: Marian Lukac marian.lukac@arm.com
@llvmbot
Copy link
Collaborator

llvmbot commented Oct 18, 2024

@llvm/pr-subscribers-llvm-globalisel
@llvm/pr-subscribers-mc

@llvm/pr-subscribers-backend-aarch64

Author: None (SpencerAbson)

Changes

note This work is based off of another change which adds the register classes and operands seen here, it is currently in review: #111717. This work has been pushed early to provide verification for the implementation of the ZK register operand, which the above patch could not provide in isolation. Thanks.

This patch adds assembly/disassembly for the following SME2p2 instructions (part of the 2024 AArch64 ISA update)

  • BFTMOPA (widening) - FEAT_SME2p2
  • BFTMOPA (non-widening) - FEAT_SME2p2 & FEAT_SME_B16B16
  • FTMOPA (4-way) - FEAT_SME2p2 & FEAT_SME_F8F32
  • FTMOPA (2-way, 8-to-16) - FEAT_SME2p2 & FEAT_SME_F8F16
  • FTMOPA (2-way, 16-to-32) - FEAT_SME2p2
  • FTMOPA (non-widening, f16) - FEAT_SME2p2 & FEAT_SME_F16F16
  • FTMOPA (non-widening, f32) - FEAT_SME2p2
    In accordance with: https://developer.arm.com/documentation/ddi0602/latest/

It also introduces .arch assembler tests for the new sme2p2 feature.
Co-authored-by: Marian Lukac marian.lukac@arm.com


Patch is 68.18 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/112876.diff

19 Files Affected:

  • (modified) llvm/lib/Target/AArch64/AArch64RegisterInfo.td (+120-28)
  • (modified) llvm/lib/Target/AArch64/AArch64SMEInstrInfo.td (+21)
  • (modified) llvm/lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp (+86-22)
  • (modified) llvm/lib/Target/AArch64/Disassembler/AArch64Disassembler.cpp (+41-2)
  • (modified) llvm/lib/Target/AArch64/MCTargetDesc/AArch64MCCodeEmitter.cpp (+28-9)
  • (modified) llvm/lib/Target/AArch64/SMEInstrFormats.td (+62)
  • (modified) llvm/test/CodeGen/AArch64/GlobalISel/regbank-inlineasm.mir (+4-4)
  • (modified) llvm/test/CodeGen/AArch64/aarch64-sve-asm.ll (+7-7)
  • (modified) llvm/test/CodeGen/AArch64/emit_fneg_with_non_register_operand.mir (+4-4)
  • (modified) llvm/test/CodeGen/AArch64/fmlal-loreg.ll (+1-1)
  • (modified) llvm/test/CodeGen/AArch64/peephole-insvigpr.mir (+2-2)
  • (added) llvm/test/MC/AArch64/SME2p2/bftmopa-diagnostics.s (+114)
  • (added) llvm/test/MC/AArch64/SME2p2/bftmopa.s (+53)
  • (added) llvm/test/MC/AArch64/SME2p2/directive-arch-negative.s (+7)
  • (added) llvm/test/MC/AArch64/SME2p2/directive-arch.s (+5)
  • (added) llvm/test/MC/AArch64/SME2p2/directive-arch_extension-negative.s (+7)
  • (added) llvm/test/MC/AArch64/SME2p2/directive-arch_extension.s (+5)
  • (added) llvm/test/MC/AArch64/SME2p2/ftmopa-diagnostics.s (+212)
  • (added) llvm/test/MC/AArch64/SME2p2/ftmopa.s (+113)
diff --git a/llvm/lib/Target/AArch64/AArch64RegisterInfo.td b/llvm/lib/Target/AArch64/AArch64RegisterInfo.td
index f754c32e1176d1..3593615a1fc204 100644
--- a/llvm/lib/Target/AArch64/AArch64RegisterInfo.td
+++ b/llvm/lib/Target/AArch64/AArch64RegisterInfo.td
@@ -923,10 +923,9 @@ class ZPRRegOp <string Suffix, AsmOperandClass C, ElementSizeEnum Size,
 //******************************************************************************
 
 // SVE predicate register classes.
-class PPRClass<int firstreg, int lastreg> : RegisterClass<
-                                  "AArch64",
+class PPRClass<int firstreg, int lastreg, int step = 1> : RegisterClass<"AArch64",
                                   [ nxv16i1, nxv8i1, nxv4i1, nxv2i1, nxv1i1 ], 16,
-                                  (sequence "P%u", firstreg, lastreg)> {
+                                  (sequence "P%u", firstreg, lastreg, step)> {
   let Size = 16;
 }
 
@@ -940,6 +939,8 @@ def PPR_p8to15 : PPRClass<8, 15> {
   let DecoderMethod = "DecodeSimpleRegisterClass<AArch64::PNRRegClassID, 8, 8>";
 }
 
+def PPRMul2 : PPRClass<0, 14, 2>;
+
 class PPRAsmOperand <string name, string RegClass, int Width>: AsmOperandClass {
   let Name = "SVE" # name # "Reg";
   let PredicateMethod = "isSVEPredicateVectorRegOfWidth<"
@@ -1098,10 +1099,11 @@ class PPRVectorListMul<int ElementWidth, int NumRegs> : PPRVectorList<ElementWid
   let DiagnosticType = "Invalid" # Name;
   let PredicateMethod =
       "isTypedVectorListMultiple<RegKind::SVEPredicateVector, " # NumRegs # ", 0, "
-                                                                # ElementWidth # ">";
+                                                                # ElementWidth #
+                                                                ", AArch64::PPRMul2RegClassID>";
 }
 
-let EncoderMethod = "EncodeRegAsMultipleOf<2>",
+let EncoderMethod = "EncodeRegMul_MinMax<2, 0, 14>",
     DecoderMethod = "DecodePPR2Mul2RegisterClass" in {
   def PP_b_mul_r : RegisterOperand<PPR2Mul2, "printTypedVectorList<0,'b'>"> {
     let ParserMatchClass = PPRVectorListMul<8, 2>;
@@ -1124,23 +1126,28 @@ let EncoderMethod = "EncodeRegAsMultipleOf<2>",
 //******************************************************************************
 
 // SVE vector register classes
-class ZPRClass<int lastreg> : RegisterClass<"AArch64",
+class ZPRClass<int firstreg, int lastreg, int step = 1> : RegisterClass<"AArch64",
                                             [nxv16i8, nxv8i16, nxv4i32, nxv2i64,
                                              nxv2f16, nxv4f16, nxv8f16,
                                              nxv2bf16, nxv4bf16, nxv8bf16,
                                              nxv2f32, nxv4f32,
                                              nxv2f64],
-                                            128, (sequence "Z%u", 0, lastreg)> {
+                                            128, (sequence "Z%u", firstreg, lastreg, step)> {
   let Size = 128;
 }
 
-def ZPR    : ZPRClass<31> {
+def ZPRMul2    : ZPRClass<0,  30, 2>;
+def ZPRMul4    : ZPRClass<0,  28, 4>;
+def ZPRMul2_Lo : ZPRClass<0,  14, 2>;
+def ZPRMul2_Hi : ZPRClass<16, 30, 2>;
+
+def ZPR    : ZPRClass<0, 31> {
   let DecoderMethod = "DecodeSimpleRegisterClass<AArch64::ZPRRegClassID, 0, 32>";
 }
-def ZPR_4b : ZPRClass<15> { // Restricted 4 bit SVE vector register class.
+def ZPR_4b : ZPRClass<0, 15> { // Restricted 4 bit SVE vector register class.
   let DecoderMethod = "DecodeSimpleRegisterClass<AArch64::ZPRRegClassID, 0, 16>";
 }
-def ZPR_3b : ZPRClass<7> {  // Restricted 3 bit SVE vector register class.
+def ZPR_3b : ZPRClass<0, 7> {  // Restricted 3 bit SVE vector register class.
   let DecoderMethod = "DecodeSimpleRegisterClass<AArch64::ZPRRegClassID, 0, 8>";
 }
 
@@ -1188,6 +1195,39 @@ def ZPR4b16 : ZPRRegOp<"h", ZPRAsmOp4b16, ElementSizeH, ZPR_4b>;
 def ZPR4b32 : ZPRRegOp<"s", ZPRAsmOp4b32, ElementSizeS, ZPR_4b>;
 def ZPR4b64 : ZPRRegOp<"d", ZPRAsmOp4b64, ElementSizeD, ZPR_4b>;
 
+class ZPRMul2_MinToMaxRegOp<string Suffix, AsmOperandClass C, int  Min, int Max, ElementSizeEnum  Width, RegisterClass RC>
+    : ZPRRegOp<Suffix, C, Width, RC> {
+  let EncoderMethod = "EncodeRegMul_MinMax<2," # Min # ", " # Max # ">";
+  let DecoderMethod = "DecodeZPRMul2_MinMax<" # Min # ", " # Max # ">";
+}
+
+def ZPRMul2AsmOp8_Lo  : ZPRAsmOperand<"VectorB_Lo",  8, "Mul2_Lo">;
+def ZPRMul2AsmOp8_Hi  : ZPRAsmOperand<"VectorB_Hi",  8, "Mul2_Hi">;
+def ZPRMul2AsmOp16_Lo : ZPRAsmOperand<"VectorH_Lo", 16, "Mul2_Lo">;
+def ZPRMul2AsmOp16_Hi : ZPRAsmOperand<"VectorH_Hi", 16, "Mul2_Hi">;
+def ZPRMul2AsmOp32_Lo : ZPRAsmOperand<"VectorS_Lo", 32, "Mul2_Lo">;
+def ZPRMul2AsmOp32_Hi : ZPRAsmOperand<"VectorS_Hi", 32, "Mul2_Hi">;
+def ZPRMul2AsmOp64_Lo : ZPRAsmOperand<"VectorD_Lo", 64, "Mul2_Lo">;
+def ZPRMul2AsmOp64_Hi : ZPRAsmOperand<"VectorD_Hi", 64, "Mul2_Hi">;
+
+def ZPR_K : RegisterClass<"AArch64", [untyped], 128,
+                          (add Z20, Z21, Z22, Z23, Z28, Z29, Z30, Z31)>;
+
+def ZK : RegisterOperand<ZPR_K, "printSVERegOp<>">{
+  let EncoderMethod = "EncodeZK";
+  let DecoderMethod = "DecodeZK";
+  let ParserMatchClass = ZPRAsmOperand<"Vector_20to23or28to31", 0, "_K">;
+}
+
+def ZPR8Mul2_Lo  : ZPRMul2_MinToMaxRegOp<"b", ZPRMul2AsmOp8_Lo,   0, 14, ElementSizeB, ZPRMul2_Lo>;
+def ZPR8Mul2_Hi  : ZPRMul2_MinToMaxRegOp<"b", ZPRMul2AsmOp8_Hi,  16, 30, ElementSizeB, ZPRMul2_Hi>;
+def ZPR16Mul2_Lo : ZPRMul2_MinToMaxRegOp<"h", ZPRMul2AsmOp16_Lo,  0, 14, ElementSizeH, ZPRMul2_Lo>;
+def ZPR16Mul2_Hi : ZPRMul2_MinToMaxRegOp<"h", ZPRMul2AsmOp16_Hi, 16, 30, ElementSizeH, ZPRMul2_Hi>;
+def ZPR32Mul2_Lo : ZPRMul2_MinToMaxRegOp<"s", ZPRMul2AsmOp32_Lo,  0, 14, ElementSizeS, ZPRMul2_Lo>;
+def ZPR32Mul2_Hi : ZPRMul2_MinToMaxRegOp<"s", ZPRMul2AsmOp32_Hi, 16, 30, ElementSizeS, ZPRMul2_Hi>;
+def ZPR64Mul2_Lo : ZPRMul2_MinToMaxRegOp<"d", ZPRMul2AsmOp64_Lo,  0, 14, ElementSizeD, ZPRMul2_Lo>;
+def ZPR64Mul2_Hi : ZPRMul2_MinToMaxRegOp<"d", ZPRMul2AsmOp64_Hi, 16, 30, ElementSizeD, ZPRMul2_Hi>;
+
 class FPRasZPR<int Width> : AsmOperandClass{
   let Name = "FPR" # Width # "asZPR";
   let PredicateMethod = "isFPRasZPR<AArch64::FPR" # Width # "RegClassID>";
@@ -1327,64 +1367,116 @@ def ZPR4Mul4 : RegisterClass<"AArch64", [untyped], 128, (add (decimate ZSeqQuads
   let Size = 512;
 }
 
-class ZPRVectorListMul<int ElementWidth, int NumRegs> : ZPRVectorList<ElementWidth, NumRegs> {
-  let Name = "SVEVectorListMul" # NumRegs # "x" # ElementWidth;
+class ZPRVectorListMul<int ElementWidth, int NumRegs, string RegClassSuffix = "">
+  : ZPRVectorList<ElementWidth, NumRegs> {
+  let Name = "SVEVectorList" # NumRegs # "x" # ElementWidth # RegClassSuffix;
   let DiagnosticType = "Invalid" # Name;
   let PredicateMethod =
-      "isTypedVectorListMultiple<RegKind::SVEDataVector, " # NumRegs # ", 0, "
-                                                           # ElementWidth # ">";
+      "isTypedVectorListMultiple<RegKind::SVEDataVector, "
+                                 # NumRegs # ", 0, "
+                                 # ElementWidth #  ", "
+                                 # "AArch64::ZPR" # RegClassSuffix # "RegClassID" # ">";
 }
 
-let EncoderMethod = "EncodeRegAsMultipleOf<2>",
-    DecoderMethod = "DecodeZPR2Mul2RegisterClass" in {
+let EncoderMethod = "EncodeRegMul_MinMax<2, 0, 30>",
+    DecoderMethod = "DecodeZPR2Mul2RegisterClass<0, 30>" in {
   def ZZ_mul_r : RegisterOperand<ZPR2Mul2, "printTypedVectorList<0,0>"> {
-    let ParserMatchClass = ZPRVectorListMul<0, 2>;
+    let ParserMatchClass = ZPRVectorListMul<0, 2, "Mul2">;
   }
 
   def ZZ_b_mul_r : RegisterOperand<ZPR2Mul2, "printTypedVectorList<0,'b'>"> {
-    let ParserMatchClass = ZPRVectorListMul<8, 2>;
+    let ParserMatchClass = ZPRVectorListMul<8, 2, "Mul2">;
   }
 
   def ZZ_h_mul_r : RegisterOperand<ZPR2Mul2, "printTypedVectorList<0,'h'>"> {
-    let ParserMatchClass = ZPRVectorListMul<16, 2>;
+    let ParserMatchClass = ZPRVectorListMul<16, 2, "Mul2">;
   }
 
   def ZZ_s_mul_r : RegisterOperand<ZPR2Mul2, "printTypedVectorList<0,'s'>"> {
-    let ParserMatchClass = ZPRVectorListMul<32, 2>;
+    let ParserMatchClass = ZPRVectorListMul<32, 2, "Mul2">;
   }
 
   def ZZ_d_mul_r : RegisterOperand<ZPR2Mul2, "printTypedVectorList<0,'d'>"> {
-    let ParserMatchClass = ZPRVectorListMul<64, 2>;
+    let ParserMatchClass = ZPRVectorListMul<64, 2, "Mul2">;
   }
 
   def ZZ_q_mul_r : RegisterOperand<ZPR2Mul2, "printTypedVectorList<0,'q'>"> {
-    let ParserMatchClass = ZPRVectorListMul<128, 2>;
+    let ParserMatchClass = ZPRVectorListMul<128, 2, "Mul2">;
   }
 } // end let EncoderMethod/DecoderMethod
 
-let EncoderMethod = "EncodeRegAsMultipleOf<4>",
+let EncoderMethod = "EncodeRegMul_MinMax<4, 0, 28>",
     DecoderMethod = "DecodeZPR4Mul4RegisterClass" in {
   def ZZZZ_b_mul_r : RegisterOperand<ZPR4Mul4, "printTypedVectorList<0,'b'>"> {
-    let ParserMatchClass = ZPRVectorListMul<8, 4>;
+    let ParserMatchClass = ZPRVectorListMul<8, 4, "Mul4">;
   }
 
   def ZZZZ_h_mul_r : RegisterOperand<ZPR4Mul4, "printTypedVectorList<0,'h'>"> {
-    let ParserMatchClass = ZPRVectorListMul<16, 4>;
+    let ParserMatchClass = ZPRVectorListMul<16, 4, "Mul4">;
   }
 
   def ZZZZ_s_mul_r : RegisterOperand<ZPR4Mul4, "printTypedVectorList<0,'s'>"> {
-    let ParserMatchClass = ZPRVectorListMul<32, 4>;
+    let ParserMatchClass = ZPRVectorListMul<32, 4, "Mul4">;
   }
 
   def ZZZZ_d_mul_r : RegisterOperand<ZPR4Mul4, "printTypedVectorList<0,'d'>"> {
-    let ParserMatchClass = ZPRVectorListMul<64, 4>;
+    let ParserMatchClass = ZPRVectorListMul<64, 4, "Mul4">;
   }
 
   def ZZZZ_q_mul_r : RegisterOperand<ZPR4Mul4, "printTypedVectorList<0,'q'>"> {
-    let ParserMatchClass = ZPRVectorListMul<128, 4>;
+    let ParserMatchClass = ZPRVectorListMul<128, 4, "Mul4">;
   }
 } // end let EncoderMethod/DecoderMethod
 
+// Pairs of ZPR, sarting with an even register, split into Lo=0-14 and Hi=16-30
+def ZPR2Mul2_Lo : RegisterClass<"AArch64", [untyped], 128,
+                                (trunc (decimate ZSeqPairs, 2), 8)>  {
+  let Size = 256;
+}
+
+def ZPR2Mul2_Hi : RegisterClass<"AArch64", [untyped], 128,
+                                (trunc (rotr (decimate ZSeqPairs, 2), 8), 8)>  {
+  let Size = 256;
+}
+
+let EncoderMethod = "EncodeRegMul_MinMax<2, 0, 14>",
+    DecoderMethod = "DecodeZPR2Mul2RegisterClass<0, 16>" in {
+  def ZZ_b_mul_r_Lo : RegisterOperand<ZPR2Mul2_Lo, "printTypedVectorList<0,'b'>"> {
+    let ParserMatchClass = ZPRVectorListMul<8, 2, "Mul2_Lo">;
+  }
+
+  def ZZ_h_mul_r_Lo : RegisterOperand<ZPR2Mul2_Lo, "printTypedVectorList<0,'h'>"> {
+    let ParserMatchClass = ZPRVectorListMul<16, 2, "Mul2_Lo">;
+  }
+
+  def ZZ_s_mul_r_Lo : RegisterOperand<ZPR2Mul2_Lo, "printTypedVectorList<0,'s'>"> {
+    let ParserMatchClass = ZPRVectorListMul<32, 2, "Mul2_Lo">;
+  }
+
+  def ZZ_d_mul_r_Lo : RegisterOperand<ZPR2Mul2_Lo, "printTypedVectorList<0,'d'>"> {
+    let ParserMatchClass = ZPRVectorListMul<64, 2, "Mul2_Lo">;
+  }
+}
+
+let EncoderMethod = "EncodeRegMul_MinMax<2, 16, 30>",
+    DecoderMethod = "DecodeZPR2Mul2RegisterClass<16, 31>" in {
+  def ZZ_b_mul_r_Hi : RegisterOperand<ZPR2Mul2_Hi, "printTypedVectorList<0,'b'>"> {
+    let ParserMatchClass = ZPRVectorListMul<8, 2, "Mul2_Hi">;
+  }
+
+  def ZZ_h_mul_r_Hi : RegisterOperand<ZPR2Mul2_Hi, "printTypedVectorList<0,'h'>"> {
+    let ParserMatchClass = ZPRVectorListMul<16, 2, "Mul2_Hi">;
+  }
+
+  def ZZ_s_mul_r_Hi : RegisterOperand<ZPR2Mul2_Hi, "printTypedVectorList<0,'s'>"> {
+    let ParserMatchClass = ZPRVectorListMul<32, 2, "Mul2_Hi">;
+  }
+
+  def ZZ_d_mul_r_Hi : RegisterOperand<ZPR2Mul2_Hi, "printTypedVectorList<0,'d'>"> {
+    let ParserMatchClass = ZPRVectorListMul<64, 2, "Mul2_Hi">;
+  }
+ } // end let EncoderMethod/DecoderMethod
+
 // SME2 strided multi-vector operands
 
 // ZStridedPairs
diff --git a/llvm/lib/Target/AArch64/AArch64SMEInstrInfo.td b/llvm/lib/Target/AArch64/AArch64SMEInstrInfo.td
index 802797a14ee42d..6044b5bb7d8151 100644
--- a/llvm/lib/Target/AArch64/AArch64SMEInstrInfo.td
+++ b/llvm/lib/Target/AArch64/AArch64SMEInstrInfo.td
@@ -1000,3 +1000,24 @@ defm FMOPA_MPPZZ_BtoS : sme_outer_product_fp32<0b0, 0b01, ZPR8, "fmopa", null_fr
 
 } //[HasSMEF8F32]
 
+let Predicates = [HasSME2p2] in {
+  def FTMOPA_M2ZZZI_HtoS  : sme_tmopa_32b<0b11000, ZZ_h_mul_r, ZPR16, "ftmopa">;
+  def FTMOPA_M2ZZZI_StoS  : sme_tmopa_32b<0b00000, ZZ_s_mul_r, ZPR32, "ftmopa">;
+  def BFTMOPA_M2ZZZI_HtoS : sme_tmopa_32b<0b10000, ZZ_h_mul_r, ZPR16, "bftmopa">;
+} // [HasSME2p2]
+
+let Predicates = [HasSME2p2, HasSMEB16B16] in {
+  def BFTMOPA_M2ZZZI_HtoH : sme_tmopa_16b<0b11001, ZZ_h_mul_r, ZPR16, "bftmopa">;
+} // [HasSME2p2, HasSMEB16B16]
+
+let Predicates = [HasSME2p2, HasSMEF8F32], Uses = [FPMR, FPCR] in {
+  def FTMOPA_M2ZZZI_BtoS : sme_tmopa_32b<0b01000, ZZ_b_mul_r, ZPR8, "ftmopa">;
+} // [HasSME2p2, HasSMEF8F32], Uses = [FPMR, FPCR]
+
+let Predicates = [HasSME2p2, HasSMEF8F16], Uses = [FPMR, FPCR] in {
+  def FTMOPA_M2ZZZI_BtoH : sme_tmopa_16b<0b01001, ZZ_b_mul_r, ZPR8, "ftmopa">;
+} // [HasSME2p2, HasSMEF8F16],  Uses = [FPMR, FPCR]
+
+let Predicates = [HasSME2p2, HasSMEF16F16] in {
+  def FTMOPA_M2ZZZI_HtoH : sme_tmopa_16b<0b10001, ZZ_h_mul_r, ZPR16, "ftmopa">;
+} // [HasSME2p2, HasSMEF16F16]
diff --git a/llvm/lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp b/llvm/lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp
index a5165d45893f3e..d0d2fda23a580b 100644
--- a/llvm/lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp
+++ b/llvm/lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp
@@ -1262,6 +1262,9 @@ class AArch64Operand : public MCParsedAsmOperand {
     case AArch64::ZPRRegClassID:
     case AArch64::ZPR_3bRegClassID:
     case AArch64::ZPR_4bRegClassID:
+    case AArch64::ZPRMul2_LoRegClassID:
+    case AArch64::ZPRMul2_HiRegClassID:
+    case AArch64::ZPR_KRegClassID:
       RK = RegKind::SVEDataVector;
       break;
     case AArch64::PPRRegClassID:
@@ -1442,13 +1445,13 @@ class AArch64Operand : public MCParsedAsmOperand {
   }
 
   template <RegKind VectorKind, unsigned NumRegs, unsigned NumElements,
-            unsigned ElementWidth>
+            unsigned ElementWidth, unsigned RegClass>
   DiagnosticPredicate isTypedVectorListMultiple() const {
     bool Res =
         isTypedVectorList<VectorKind, NumRegs, NumElements, ElementWidth>();
     if (!Res)
       return DiagnosticPredicateTy::NoMatch;
-    if (((VectorList.RegNum - AArch64::Z0) % NumRegs) != 0)
+    if (!AArch64MCRegisterClasses[RegClass].contains(VectorList.RegNum))
       return DiagnosticPredicateTy::NearMatch;
     return DiagnosticPredicateTy::Match;
   }
@@ -6092,6 +6095,33 @@ bool AArch64AsmParser::showMatchError(SMLoc Loc, unsigned ErrCode,
     return Error(Loc, "Invalid restricted vector register, expected z0.s..z15.s");
   case Match_InvalidZPR_4b64:
     return Error(Loc, "Invalid restricted vector register, expected z0.d..z15.d");
+  case Match_InvalidZPRMul2_Lo8:
+    return Error(Loc, "Invalid restricted vector register, expected even "
+                      "register in z0.b..z14.b");
+  case Match_InvalidZPRMul2_Hi8:
+    return Error(Loc, "Invalid restricted vector register, expected even "
+                      "register in z16.b..z30.b");
+  case Match_InvalidZPRMul2_Lo16:
+    return Error(Loc, "Invalid restricted vector register, expected even "
+                      "register in z0.h..z14.h");
+  case Match_InvalidZPRMul2_Hi16:
+    return Error(Loc, "Invalid restricted vector register, expected even "
+                      "register in z16.h..z30.h");
+  case Match_InvalidZPRMul2_Lo32:
+    return Error(Loc, "Invalid restricted vector register, expected even "
+                      "register in z0.s..z14.s");
+  case Match_InvalidZPRMul2_Hi32:
+    return Error(Loc, "Invalid restricted vector register, expected even "
+                      "register in z16.s..z30.s");
+  case Match_InvalidZPRMul2_Lo64:
+    return Error(Loc, "Invalid restricted vector register, expected even "
+                      "register in z0.d..z14.d");
+  case Match_InvalidZPRMul2_Hi64:
+    return Error(Loc, "Invalid restricted vector register, expected even "
+                      "register in z16.d..z30.d");
+  case Match_InvalidZPR_K0:
+    return Error(Loc, "invalid restricted vector register, expected register "
+                      "in z20..z23 or z28..z31");
   case Match_InvalidSVEPattern:
     return Error(Loc, "invalid predicate pattern");
   case Match_InvalidSVEPPRorPNRAnyReg:
@@ -6171,19 +6201,36 @@ bool AArch64AsmParser::showMatchError(SMLoc Loc, unsigned ErrCode,
     return Error(Loc, "operand must be a register in range [w12, w15]");
   case Match_InvalidMatrixIndexGPR32_8_11:
     return Error(Loc, "operand must be a register in range [w8, w11]");
-  case Match_InvalidSVEVectorListMul2x8:
-  case Match_InvalidSVEVectorListMul2x16:
-  case Match_InvalidSVEVectorListMul2x32:
-  case Match_InvalidSVEVectorListMul2x64:
-  case Match_InvalidSVEVectorListMul2x128:
+  case Match_InvalidSVEVectorList2x8Mul2:
+  case Match_InvalidSVEVectorList2x16Mul2:
+  case Match_InvalidSVEVectorList2x32Mul2:
+  case Match_InvalidSVEVectorList2x64Mul2:
+  case Match_InvalidSVEVectorList2x128Mul2:
     return Error(Loc, "Invalid vector list, expected list with 2 consecutive "
                       "SVE vectors, where the first vector is a multiple of 2 "
                       "and with matching element types");
-  case Match_InvalidSVEVectorListMul4x8:
-  case Match_InvalidSVEVectorListMul4x16:
-  case Match_InvalidSVEVectorListMul4x32:
-  case Match_InvalidSVEVectorListMul4x64:
-  case Match_InvalidSVEVectorListMul4x128:
+  case Match_InvalidSVEVectorList2x8Mul2_Lo:
+  case Match_InvalidSVEVectorList2x16Mul2_Lo:
+  case Match_InvalidSVEVectorList2x32Mul2_Lo:
+  case Match_InvalidSVEVectorList2x64Mul2_Lo:
+    return Error(Loc, "Invalid vector list, expected list with 2 consecutive "
+                      "SVE vectors in the range z0-z14, where the first vector "
+                      "is a multiple of 2 "
+                      "and with matching element types");
+  case Match_InvalidSVEVectorList2x8Mul2_Hi:
+  case Match_InvalidSVEVectorList2x16Mul2_Hi:
+  case Match_InvalidSVEVectorList2x32Mul2_Hi:
+  case Match_InvalidSVEVectorList2x64Mul2_Hi:
+    return Error(Loc,
+                 "Invalid vector list, expected list with 2 consecutive "
+                 "SVE vectors in the range z16-z30, where the first vector "
+                 "is a multiple of 2 "
+                 "and with matching element types");
+  case Match_InvalidSVEVectorList4x8Mul4:
+  case Match_InvalidSVEVectorList4x16Mul4:
+  case Match_InvalidSVEVectorList4x32Mul4:
+  case Match_InvalidSVEVectorList4x64Mul4:
+  case Match_InvalidSVEVectorList4x128Mul4:
     return Error(Loc, "Invalid vector list, expected list with 4 consecutive "
                       "SVE vectors, where the first vector is a multiple of 4 "
                       "and with matching element types");
@@ -6776,16 +6823,33 @@ bool AArch64AsmParser::matchAndEmitInstruction(SMLoc IDLoc, unsigned &Opcode,
   case Match_InvalidMatrixIndexGPR32_12_15:
   case Match_InvalidMatrixIndexGPR32_8_11:
   case Match_InvalidLookupTable:
-  case Match_InvalidSVEVectorListMul2x8:
-  case Match_InvalidSVEVectorListMul2x16:
-  case Match_InvalidSVEVectorListMul2x32:
-  case Match_InvalidSVEVectorListMul2x64:
-  case Match_InvalidSVEVectorListMul2x128:
-  case Match_InvalidSVEVectorListMul4x8:
-  case Match_InvalidSVEVectorListMul4x16:
-  case Match_InvalidSVEVectorListMul4x32:
-  case Match_InvalidSVEVectorListMul4x64:
-  case Match_InvalidSVEVectorListMul4x128:
+  case Match_InvalidZPRMul2_Lo8:
+  case Match_InvalidZPRMul2_Hi8:
+  case Match_InvalidZPRMul2_Lo16:
+  case Match_InvalidZPRMul2_Hi16:
+  case Match_InvalidZPRMul2_Lo32:
+  case Match_InvalidZPRMul2_Hi32:
+  case Match_InvalidZPRMul2_Lo64:
+  case Match_InvalidZPRMul2_Hi64:
+  case Match_InvalidZPR_K0:
+  case Match_InvalidSVEVectorList2x8Mul2:
+  case Match_InvalidSVEVectorList2x16Mul2:
+  case Match_InvalidSVEVectorList2x32Mul2:
+  case Match_InvalidSVEVectorList2x64Mul2:
+  case Match_InvalidSVEVectorList2x128Mul2:
+  case Match_InvalidSVEVectorList4x8Mul4:
+  case Match_InvalidSVEVectorList4x16Mul4:
+  case Match_InvalidSVEVectorList4x32Mul4:
+  case Match_InvalidSVEVectorList4x64Mul4:
+  case Match_InvalidSVEVectorList4x128Mul4:
+  case Match_InvalidSVEVectorList2x8Mul2_Lo:
+  case Match_InvalidSVEVectorList2x16Mul2_Lo:
+  case Ma...
[truncated]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants