Skip to content

[llvm][RISCV] Support RISCV vector tuple CodeGen and Calling Convention #97995

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

4vtomat
Copy link
Member

@4vtomat 4vtomat commented Jul 8, 2024

This patch handles target lowering and calling convention.

For target lowering, the vector tuple type represented as multiple
scalable vectors is now changed to a single MVT, each MVT has a
corresponding register class.

The load/store of vector tuples are handled as the same way but need
another vector insert/extract instructions to get sub-register group.

Inline assembly constraint for vector tuple type can directly be modeled
as "vr" which is identical to normal vector registers.

For calling convention, it no longer needs an alternative algorithm to
handle register allocation, this makes the code easier to maintain and
read.

Stacked on #97994

@llvmbot llvmbot added clang Clang issues not falling into any other category backend:RISC-V clang:frontend Language frontend issues, e.g. anything involving "Sema" clang:codegen IR generation bugs: mangling, exceptions, etc. llvm:SelectionDAG SelectionDAGISel as well llvm:ir llvm:analysis llvm:transforms labels Jul 8, 2024
@llvmbot
Copy link
Member

llvmbot commented Jul 8, 2024

@llvm/pr-subscribers-clang
@llvm/pr-subscribers-clang-codegen
@llvm/pr-subscribers-llvm-transforms
@llvm/pr-subscribers-llvm-analysis
@llvm/pr-subscribers-backend-risc-v

@llvm/pr-subscribers-llvm-selectiondag

Author: Brandon Wu (4vtomat)

Changes
  • [llvm][RISCV] Support RISCV vector tuple type in llvm IR
  • [llvm][RISCV] Add RISCV vector tuple type to value types(MVT)
  • Revert "Recommit [RISCV] RISCV vector calling convention (2/2) (#79096) (#87736)"
  • [llvm][RISCV] Support RISCV vector tuple CodeGen and Calling Convention

Patch is 113.69 MiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/97995.diff

1056 Files Affected:

  • (modified) clang/include/clang/Basic/riscv_vector.td (+107-90)
  • (modified) clang/include/clang/Support/RISCVVIntrinsicUtils.h (+1)
  • (modified) clang/lib/CodeGen/CGBuiltin.cpp (+2-1)
  • (modified) clang/lib/CodeGen/CodeGenTypes.cpp (+18-7)
  • (modified) clang/lib/Support/RISCVVIntrinsicUtils.cpp (+1-1)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vcreate.c (+166-166)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vget.c (+50-50)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vloxseg2ei16.c (+30-30)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vloxseg3ei16.c (+24-24)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vloxseg4ei16.c (+24-24)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vloxseg5ei16.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vloxseg6ei16.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vloxseg7ei16.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vloxseg8ei16.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlseg2e16.c (+30-30)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlseg2e16ff.c (+60-90)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlseg3e16.c (+24-24)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlseg3e16ff.c (+48-88)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlseg4e16.c (+24-24)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlseg4e16ff.c (+48-104)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlseg5e16.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlseg5e16ff.c (+36-90)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlseg6e16.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlseg6e16ff.c (+36-102)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlseg7e16.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlseg7e16ff.c (+36-114)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlseg8e16.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlseg8e16ff.c (+36-126)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlsseg2e16.c (+30-30)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlsseg3e16.c (+24-24)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlsseg4e16.c (+24-24)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlsseg5e16.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlsseg6e16.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlsseg7e16.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vlsseg8e16.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vluxseg2ei16.c (+30-30)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vluxseg3ei16.c (+24-24)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vluxseg4ei16.c (+24-24)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vluxseg5ei16.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vluxseg6ei16.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vluxseg7ei16.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vluxseg8ei16.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vset.c (+100-100)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsoxseg2ei16.c (+20-40)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsoxseg3ei16.c (+16-40)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsoxseg4ei16.c (+16-48)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsoxseg5ei16.c (+12-42)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsoxseg6ei16.c (+12-48)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsoxseg7ei16.c (+12-54)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsoxseg8ei16.c (+12-60)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsseg2e16.c (+20-40)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsseg3e16.c (+16-40)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsseg4e16.c (+16-48)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsseg5e16.c (+12-42)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsseg6e16.c (+12-48)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsseg7e16.c (+12-54)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsseg8e16.c (+12-60)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vssseg2e16.c (+20-40)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vssseg3e16.c (+16-40)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vssseg4e16.c (+16-48)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vssseg5e16.c (+12-42)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vssseg6e16.c (+12-48)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vssseg7e16.c (+12-54)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vssseg8e16.c (+12-60)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsuxseg2ei16.c (+20-40)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsuxseg3ei16.c (+16-40)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsuxseg4ei16.c (+16-48)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsuxseg5ei16.c (+12-42)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsuxseg6ei16.c (+12-48)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsuxseg7ei16.c (+12-54)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vsuxseg8ei16.c (+12-60)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/bfloat16/vundefined.c (+50-50)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vcreate.c (+1483-1483)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vget.c (+452-452)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg2ei16.c (+288-288)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg2ei32.c (+276-276)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg2ei64.c (+246-246)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg2ei8.c (+288-288)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg3ei16.c (+222-222)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg3ei32.c (+222-222)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg3ei64.c (+210-210)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg3ei8.c (+222-222)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg4ei16.c (+222-222)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg4ei32.c (+222-222)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg4ei64.c (+210-210)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg4ei8.c (+222-222)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg5ei16.c (+156-156)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg5ei32.c (+156-156)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg5ei64.c (+156-156)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg5ei8.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg6ei16.c (+156-156)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg6ei32.c (+156-156)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg6ei64.c (+156-156)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg6ei8.c (+156-156)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg7ei16.c (+156-156)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg7ei32.c (+156-156)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg7ei64.c (+156-156)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg7ei8.c (+156-156)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg8ei16.c (+156-156)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg8ei32.c (+156-156)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg8ei64.c (+156-156)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vloxseg8ei8.c (+156-156)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg2e16.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg2e16ff.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg2e32.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg2e32ff.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg2e64.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg2e64ff.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg2e8.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg2e8ff.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg3e16.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg3e16ff.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg3e32.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg3e32ff.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg3e64.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg3e64ff.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg3e8.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg3e8ff.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg4e16.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg4e16ff.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg4e32.c (+54-54)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg4e32ff.c (+108-234)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg4e64.c (+36-36)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg4e64ff.c (+72-156)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg4e8.c (+60-60)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg4e8ff.c (+120-260)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg5e16.c (+54-54)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg5e16ff.c (+108-270)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg5e32.c (+36-36)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg5e32ff.c (+72-180)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg5e64.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg5e64ff.c (+36-90)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg5e8.c (+48-48)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg5e8ff.c (+96-240)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg6e16.c (+54-54)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg6e16ff.c (+108-306)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg6e32.c (+36-36)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg6e32ff.c (+72-204)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg6e64.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg6e64ff.c (+36-102)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg6e8.c (+48-48)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg6e8ff.c (+96-272)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg7e16.c (+54-54)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg7e16ff.c (+108-342)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg7e32.c (+36-36)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg7e32ff.c (+72-228)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg7e64.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg7e64ff.c (+36-114)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg7e8.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg7e8ff.c ()
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg8e16.c (+54-54)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg8e16ff.c (+108-378)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg8e32.c (+36-36)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg8e32ff.c (+72-252)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg8e64.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg8e64ff.c (+36-126)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg8e8.c (+48-48)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlseg8e8ff.c (+96-336)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlsseg2e16.c (+90-90)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlsseg2e32.c (+72-72)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlsseg2e64.c (+54-54)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlsseg2e8.c (+72-72)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlsseg3e16.c (+72-72)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlsseg3e32.c (+54-54)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlsseg3e64.c (+36-36)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlsseg3e8.c (+60-60)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlsseg4e16.c (+72-72)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlsseg4e32.c (+54-54)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlsseg4e64.c (+36-36)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlsseg4e8.c (+60-60)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlsseg5e16.c (+54-54)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlsseg5e32.c (+36-36)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlsseg5e64.c (+18-18)
  • (modified) clang/test/CodeGen/RISCV/rvv-intrinsics-autogenerated/non-policy/non-overloaded/vlsseg5e8.c (+48-48)
diff --git a/clang/include/clang/Basic/riscv_vector.td b/clang/include/clang/Basic/riscv_vector.td
index a0820e2093bc20..67f480dec0fe3f 100644
--- a/clang/include/clang/Basic/riscv_vector.td
+++ b/clang/include/clang/Basic/riscv_vector.td
@@ -762,8 +762,10 @@ multiclass RVVUnitStridedSegLoadTuple<string op> {
                                                    []<string>)),
             ManualCodegen = [{
     {
-      llvm::Type *ElementVectorType = cast<StructType>(ResultType)->elements()[0];
-      IntrinsicTypes = {ElementVectorType, Ops.back()->getType()};
+      if (IsMasked)
+        IntrinsicTypes = {ResultType, Ops[0]->getType(), Ops.back()->getType()};
+      else
+        IntrinsicTypes = {ResultType, Ops.back()->getType()};
       SmallVector<llvm::Value*, 12> Operands;
 
       bool NoPassthru =
@@ -772,11 +774,10 @@ multiclass RVVUnitStridedSegLoadTuple<string op> {
       unsigned Offset = IsMasked ? NoPassthru ? 1 : 2 : NoPassthru ? 0 : 1;
 
       if (NoPassthru) { // Push poison into passthru
-        Operands.append(NF, llvm::PoisonValue::get(ElementVectorType));
+        Operands.push_back(llvm::PoisonValue::get(ResultType));
       } else { // Push intrinsics operands into passthru
         llvm::Value *PassthruOperand = IsMasked ? Ops[1] : Ops[0];
-        for (unsigned I = 0; I < NF; ++I)
-          Operands.push_back(Builder.CreateExtractValue(PassthruOperand, {I}));
+        Operands.push_back(PassthruOperand);
       }
 
       Operands.push_back(Ops[Offset]); // Ptr
@@ -785,6 +786,7 @@ multiclass RVVUnitStridedSegLoadTuple<string op> {
       Operands.push_back(Ops[Offset + 1]); // VL
       if (IsMasked)
         Operands.push_back(ConstantInt::get(Ops.back()->getType(), PolicyAttrs));
+      Operands.push_back(ConstantInt::get(Ops.back()->getType(), SegInstSEW));
 
       llvm::Function *F = CGM.getIntrinsic(ID, IntrinsicTypes);
 
@@ -828,24 +830,24 @@ multiclass RVVUnitStridedSegStoreTuple<string op> {
     {
       // Masked
       // Builtin: (mask, ptr, v_tuple, vl)
-      // Intrinsic: (val0, val1, ..., ptr, mask, vl)
+      // Intrinsic: (tuple, ptr, mask, vl)
       // Unmasked
       // Builtin: (ptr, v_tuple, vl)
-      // Intrinsic: (val0, val1, ..., ptr, vl)
+      // Intrinsic: (tuple, ptr, vl)
       unsigned Offset = IsMasked ? 1 : 0;
-      llvm::Value *VTupleOperand = Ops[Offset + 1];
 
       SmallVector<llvm::Value*, 12> Operands;
-      for (unsigned I = 0; I < NF; ++I) {
-        llvm::Value *V = Builder.CreateExtractValue(VTupleOperand, {I});
-        Operands.push_back(V);
-      }
+      Operands.push_back(Ops[Offset + 1]); // tuple
       Operands.push_back(Ops[Offset]); // Ptr
       if (IsMasked)
         Operands.push_back(Ops[0]);
       Operands.push_back(Ops[Offset + 2]); // VL
+      Operands.push_back(ConstantInt::get(Ops.back()->getType(), SegInstSEW));
 
-      IntrinsicTypes = {Operands[0]->getType(), Operands.back()->getType()};
+      if (IsMasked)
+        IntrinsicTypes = {Operands[0]->getType(), Ops[0]->getType(), Operands.back()->getType()};
+      else
+        IntrinsicTypes = {Operands[0]->getType(), Operands.back()->getType()};
       llvm::Function *F = CGM.getIntrinsic(ID, IntrinsicTypes);
       return Builder.CreateCall(F, Operands, "");
    }
@@ -880,8 +882,10 @@ multiclass RVVUnitStridedSegLoadFFTuple<string op> {
                                                    []<string>)),
             ManualCodegen = [{
     {
-      llvm::Type *ElementVectorType = cast<StructType>(ResultType)->elements()[0];
-      IntrinsicTypes = {ElementVectorType, Ops.back()->getType()};
+      if (IsMasked)
+        IntrinsicTypes = {ResultType, Ops.back()->getType(), Ops[0]->getType()};
+      else
+        IntrinsicTypes = {ResultType, Ops.back()->getType()};
       SmallVector<llvm::Value*, 12> Operands;
 
       bool NoPassthru =
@@ -890,11 +894,10 @@ multiclass RVVUnitStridedSegLoadFFTuple<string op> {
       unsigned Offset = IsMasked ? NoPassthru ? 1 : 2 : NoPassthru ? 0 : 1;
 
       if (NoPassthru) { // Push poison into passthru
-        Operands.append(NF, llvm::PoisonValue::get(ElementVectorType));
+        Operands.push_back(llvm::PoisonValue::get(ResultType));
       } else { // Push intrinsics operands into passthru
         llvm::Value *PassthruOperand = IsMasked ? Ops[1] : Ops[0];
-        for (unsigned I = 0; I < NF; ++I)
-          Operands.push_back(Builder.CreateExtractValue(PassthruOperand, {I}));
+        Operands.push_back(PassthruOperand);
       }
 
       Operands.push_back(Ops[Offset]); // Ptr
@@ -903,6 +906,7 @@ multiclass RVVUnitStridedSegLoadFFTuple<string op> {
       Operands.push_back(Ops[Offset + 2]); // vl
       if (IsMasked)
         Operands.push_back(ConstantInt::get(Ops.back()->getType(), PolicyAttrs));
+      Operands.push_back(ConstantInt::get(Ops.back()->getType(), SegInstSEW));
 
       llvm::Function *F = CGM.getIntrinsic(ID, IntrinsicTypes);
 
@@ -911,14 +915,10 @@ multiclass RVVUnitStridedSegLoadFFTuple<string op> {
       clang::CharUnits Align =
           CGM.getNaturalPointeeTypeAlignment(E->getArg(Offset + 1)->getType());
 
-      llvm::Value *ReturnTuple = llvm::PoisonValue::get(ResultType);
-      for (unsigned I = 0; I < NF; ++I) {
-        llvm::Value *V = Builder.CreateExtractValue(LoadValue, {I});
-        ReturnTuple = Builder.CreateInsertValue(ReturnTuple, V, {I});
-      }
+      llvm::Value *ReturnTuple = Builder.CreateExtractValue(LoadValue, 0);
 
       // Store new_vl
-      llvm::Value *V = Builder.CreateExtractValue(LoadValue, {NF});
+      llvm::Value *V = Builder.CreateExtractValue(LoadValue, 1);
       Builder.CreateStore(V, Address(Ops[Offset + 1], V->getType(), Align));
 
       if (ReturnValue.isNull())
@@ -957,8 +957,10 @@ multiclass RVVStridedSegLoadTuple<string op> {
                                                    []<string>)),
             ManualCodegen = [{
     {
-      llvm::Type *ElementVectorType = cast<StructType>(ResultType)->elements()[0];
-      IntrinsicTypes = {ElementVectorType, Ops.back()->getType()};
+      if (IsMasked)
+        IntrinsicTypes = {ResultType, Ops.back()->getType(), Ops[0]->getType()};
+      else
+        IntrinsicTypes = {ResultType, Ops.back()->getType()};
       SmallVector<llvm::Value*, 12> Operands;
 
       bool NoPassthru =
@@ -967,11 +969,10 @@ multiclass RVVStridedSegLoadTuple<string op> {
       unsigned Offset = IsMasked ? NoPassthru ? 1 : 2 : NoPassthru ? 0 : 1;
 
       if (NoPassthru) { // Push poison into passthru
-        Operands.append(NF, llvm::PoisonValue::get(ElementVectorType));
+        Operands.push_back(llvm::PoisonValue::get(ResultType));
       } else { // Push intrinsics operands into passthru
         llvm::Value *PassthruOperand = IsMasked ? Ops[1] : Ops[0];
-        for (unsigned I = 0; I < NF; ++I)
-          Operands.push_back(Builder.CreateExtractValue(PassthruOperand, {I}));
+        Operands.push_back(PassthruOperand);
       }
 
       Operands.push_back(Ops[Offset]); // Ptr
@@ -981,6 +982,7 @@ multiclass RVVStridedSegLoadTuple<string op> {
       Operands.push_back(Ops[Offset + 2]); // VL
       if (IsMasked)
         Operands.push_back(ConstantInt::get(Ops.back()->getType(), PolicyAttrs));
+      Operands.push_back(ConstantInt::get(Ops.back()->getType(), SegInstSEW));
 
       llvm::Function *F = CGM.getIntrinsic(ID, IntrinsicTypes);
       llvm::Value *LoadValue = Builder.CreateCall(F, Operands, "");
@@ -1025,25 +1027,25 @@ multiclass RVVStridedSegStoreTuple<string op> {
     {
       // Masked
       // Builtin: (mask, ptr, stride, v_tuple, vl)
-      // Intrinsic: (val0, val1, ..., ptr, stride, mask, vl)
+      // Intrinsic: (tuple, ptr, stride, mask, vl)
       // Unmasked
       // Builtin: (ptr, stride, v_tuple, vl)
-      // Intrinsic: (val0, val1, ..., ptr, stride, vl)
+      // Intrinsic: (tuple, ptr, stride, vl)
       unsigned Offset = IsMasked ? 1 : 0;
-      llvm::Value *VTupleOperand = Ops[Offset + 2];
 
       SmallVector<llvm::Value*, 12> Operands;
-      for (unsigned I = 0; I < NF; ++I) {
-        llvm::Value *V = Builder.CreateExtractValue(VTupleOperand, {I});
-        Operands.push_back(V);
-      }
+      Operands.push_back(Ops[Offset + 2]); // tuple
       Operands.push_back(Ops[Offset]); // Ptr
       Operands.push_back(Ops[Offset + 1]); // Stride
       if (IsMasked)
         Operands.push_back(Ops[0]);
       Operands.push_back(Ops[Offset + 3]); // VL
+      Operands.push_back(ConstantInt::get(Ops.back()->getType(), SegInstSEW));
 
-      IntrinsicTypes = {Operands[0]->getType(), Operands.back()->getType()};
+      if (IsMasked)
+        IntrinsicTypes = {Operands[0]->getType(), Operands.back()->getType(), Ops[0]->getType()};
+      else
+        IntrinsicTypes = {Operands[0]->getType(), Operands.back()->getType()};
       llvm::Function *F = CGM.getIntrinsic(ID, IntrinsicTypes);
       return Builder.CreateCall(F, Operands, "");
     }
@@ -1073,8 +1075,6 @@ multiclass RVVIndexedSegLoadTuple<string op> {
                                                    []<string>)),
             ManualCodegen = [{
     {
-      llvm::Type *ElementVectorType = cast<StructType>(ResultType)->elements()[0];
-      IntrinsicTypes = {ElementVectorType, Ops.back()->getType()};
       SmallVector<llvm::Value*, 12> Operands;
 
       bool NoPassthru =
@@ -1083,11 +1083,10 @@ multiclass RVVIndexedSegLoadTuple<string op> {
       unsigned Offset = IsMasked ? NoPassthru ? 1 : 2 : NoPassthru ? 0 : 1;
 
       if (NoPassthru) { // Push poison into passthru
-        Operands.append(NF, llvm::PoisonValue::get(ElementVectorType));
+        Operands.push_back(llvm::PoisonValue::get(ResultType));
       } else { // Push intrinsics operands into passthru
         llvm::Value *PassthruOperand = IsMasked ? Ops[1] : Ops[0];
-        for (unsigned I = 0; I < NF; ++I)
-          Operands.push_back(Builder.CreateExtractValue(PassthruOperand, {I}));
+        Operands.push_back(PassthruOperand);
       }
 
       Operands.push_back(Ops[Offset]); // Ptr
@@ -1097,9 +1096,15 @@ multiclass RVVIndexedSegLoadTuple<string op> {
       Operands.push_back(Ops[Offset + 2]); // VL
       if (IsMasked)
         Operands.push_back(ConstantInt::get(Ops.back()->getType(), PolicyAttrs));
+      Operands.push_back(ConstantInt::get(Ops.back()->getType(), SegInstSEW));
 
-      IntrinsicTypes = {ElementVectorType, Ops[Offset + 1]->getType(),
-                        Ops.back()->getType()};
+      if (IsMasked)
+        IntrinsicTypes = {ResultType, Ops[Offset + 1]->getType(),
+                          Ops[0]->getType(),
+                          Ops.back()->getType()};
+      else
+        IntrinsicTypes = {ResultType, Ops[Offset + 1]->getType(),
+                          Ops.back()->getType()};
       llvm::Function *F = CGM.getIntrinsic(ID, IntrinsicTypes);
       llvm::Value *LoadValue = Builder.CreateCall(F, Operands, "");
 
@@ -1139,26 +1144,28 @@ multiclass RVVIndexedSegStoreTuple<string op> {
     {
       // Masked
       // Builtin: (mask, ptr, index, v_tuple, vl)
-      // Intrinsic: (val0, val1, ..., ptr, index, mask, vl)
+      // Intrinsic: (tuple, ptr, index, mask, vl)
       // Unmasked
       // Builtin: (ptr, index, v_tuple, vl)
-      // Intrinsic: (val0, val1, ..., ptr, index, vl)
+      // Intrinsic: (tuple, ptr, index, vl)
       unsigned Offset = IsMasked ? 1 : 0;
-      llvm::Value *VTupleOperand = Ops[Offset + 2];
 
       SmallVector<llvm::Value*, 12> Operands;
-      for (unsigned I = 0; I < NF; ++I) {
-        llvm::Value *V = Builder.CreateExtractValue(VTupleOperand, {I});
-        Operands.push_back(V);
-      }
+      Operands.push_back(Ops[Offset + 2]); // tuple
       Operands.push_back(Ops[Offset]); // Ptr
       Operands.push_back(Ops[Offset + 1]); // Idx
       if (IsMasked)
         Operands.push_back(Ops[0]);
       Operands.push_back(Ops[Offset + 3]); // VL
+      Operands.push_back(ConstantInt::get(Ops.back()->getType(), SegInstSEW));
 
-      IntrinsicTypes = {Operands[0]->getType(), Ops[Offset + 1]->getType(),
-                        Operands.back()->getType()};
+      if (IsMasked)
+        IntrinsicTypes = {Operands[0]->getType(), Ops[Offset + 1]->getType(),
+                          Ops[0]->getType(),
+                          Operands.back()->getType()};
+      else
+        IntrinsicTypes = {Operands[0]->getType(), Ops[Offset + 1]->getType(),
+                          Operands.back()->getType()};
       llvm::Function *F = CGM.getIntrinsic(ID, IntrinsicTypes);
       return Builder.CreateCall(F, Operands, "");
     }
@@ -2468,22 +2475,25 @@ let HasMasked = false, HasVL = false, IRName = "" in {
   let Name = "vget_v", MaskedPolicyScheme = NonePolicy,
       ManualCodegen = [{
       {
-        if (isa<StructType>(Ops[0]->getType())) // For tuple type
-          // Extract value from index (operand 1) of vtuple (operand 0)
-          return Builder.CreateExtractValue(
-            Ops[0],
-            {(unsigned)cast<ConstantInt>(Ops[1])->getZExtValue()});
         auto *VecTy = cast<ScalableVectorType>(ResultType);
-        auto *OpVecTy = cast<ScalableVectorType>(Ops[0]->getType());
         // Mask to only valid indices.
-        unsigned MaxIndex = OpVecTy->getMinNumElements() / VecTy->getMinNumElements();
-        assert(isPowerOf2_32(MaxIndex));
         Ops[1] = Builder.CreateZExt(Ops[1], Builder.getInt64Ty());
-        Ops[1] = Builder.CreateAnd(Ops[1], MaxIndex - 1);
-        Ops[1] = Builder.CreateMul(Ops[1],
-                                   ConstantInt::get(Ops[1]->getType(),
-                                                    VecTy->getMinNumElements()));
-        return Builder.CreateExtractVector(ResultType, Ops[0], Ops[1]);
+        if (auto *OpVecTy = dyn_cast<ScalableVectorType>(Ops[0]->getType())) {
+          unsigned MaxIndex = OpVecTy->getMinNumElements() / VecTy->getMinNumElements();
+          assert(isPowerOf2_32(MaxIndex));
+          Ops[1] = Builder.CreateAnd(Ops[1], MaxIndex - 1);
+          Ops[1] = Builder.CreateMul(Ops[1],
+                                     ConstantInt::get(Ops[1]->getType(),
+                                                      VecTy->getMinNumElements()));
+          return Builder.CreateExtractVector(ResultType, Ops[0], Ops[1]);
+        }
+
+        bool IsRISCV64 = getTarget().getTriple().isRISCV64();
+        llvm::Type *XLenTy = IsRISCV64 ? Builder.getInt64Ty() :
+                                         Builder.getInt32Ty();
+        return Builder.CreateIntrinsic(Intrinsic::riscv_vector_extract,
+                                       {ResultType, Ops[0]->getType(), XLenTy},
+                                       {Ops[0], Ops[1]});
       }
       }] in {
     foreach dst_lmul = ["(SFixedLog2LMUL:0)", "(SFixedLog2LMUL:1)", "(SFixedLog2LMUL:2)"] in {
@@ -2500,22 +2510,25 @@ let HasMasked = false, HasVL = false, IRName = "" in {
   let Name = "vset_v", MaskedPolicyScheme = NonePolicy,
       ManualCodegen = [{
       {
-        if (isa<StructType>(ResultType)) // For tuple type
-          // Insert value (operand 2) into index (operand 1) of vtuple (operand 0)
-          return Builder.CreateInsertValue(
-            Ops[0], Ops[2],
-            {(unsigned)cast<ConstantInt>(Ops[1])->getZExtValue()});
-        auto *ResVecTy = cast<ScalableVectorType>(ResultType);
         auto *VecTy = cast<ScalableVectorType>(Ops[2]->getType());
         // Mask to only valid indices.
-        unsigned MaxIndex = ResVecTy->getMinNumElements() / VecTy->getMinNumElements();
-        assert(isPowerOf2_32(MaxIndex));
         Ops[1] = Builder.CreateZExt(Ops[1], Builder.getInt64Ty());
-        Ops[1] = Builder.CreateAnd(Ops[1], MaxIndex - 1);
-        Ops[1] = Builder.CreateMul(Ops[1],
-                                   ConstantInt::get(Ops[1]->getType(),
-                                                    VecTy->getMinNumElements()));
-        return Builder.CreateInsertVector(ResultType, Ops[0], Ops[2], Ops[1]);
+        if (auto *ResVecTy = dyn_cast<ScalableVectorType>(ResultType)) {
+          unsigned MaxIndex = ResVecTy->getMinNumElements() / VecTy->getMinNumElements();
+          assert(isPowerOf2_32(MaxIndex));
+          Ops[1] = Builder.CreateAnd(Ops[1], MaxIndex - 1);
+          Ops[1] = Builder.CreateMul(Ops[1],
+                                     ConstantInt::get(Ops[1]->getType(),
+                                                      VecTy->getMinNumElements()));
+          return Builder.CreateInsertVector(ResultType, Ops[0], Ops[2], Ops[1]);
+        }
+
+        bool IsRISCV64 = getTarget().getTriple().isRISCV64();
+        llvm::Type *XLenTy = IsRISCV64 ? Builder.getInt64Ty() :
+                                         Builder.getInt32Ty();
+        return Builder.CreateIntrinsic(Intrinsic::riscv_vector_insert,
+                                       {ResultType, Ops[2]->getType(), XLenTy},
+                                       {Ops[0], Ops[2], Ops[1]});
       }
       }] in {
     foreach dst_lmul = ["(LFixedLog2LMUL:1)", "(LFixedLog2LMUL:2)", "(LFixedLog2LMUL:3)"] in {
@@ -2539,22 +2552,26 @@ let HasMasked = false, HasVL = false, IRName = "" in {
       SupportOverloading = false,
       ManualCodegen = [{
       {
-        if (isa<StructType>(ResultType)) {
-          unsigned NF = cast<StructType>(ResultType)->getNumElements();
-          llvm::Value *ReturnTuple = llvm::PoisonValue::get(ResultType);
-          for (unsigned I = 0; I < NF; ++I) {
-            ReturnTuple = Builder.CreateInsertValue(ReturnTuple, Ops[I], {I});
-          }
-          return ReturnTuple;
-        }
         llvm::Value *ReturnVector = llvm::PoisonValue::get(ResultType);
         auto *VecTy = cast<ScalableVectorType>(Ops[0]->getType());
+        bool IsRISCV64 = getTarget().getTriple().isRISCV64();
+        llvm::Type *XLenTy = IsRISCV64 ? Builder.getInt64Ty() :
+                                         Builder.getInt32Ty();
         for (unsigned I = 0, N = Ops.size(); I < N; ++I) {
           llvm::Value *Idx =
             ConstantInt::get(Builder.getInt64Ty(),
-                              VecTy->getMinNumElements() * I);
-          ReturnVector =
-            Builder.CreateInsertVector(ResultType, ReturnVector, Ops[I], Idx);
+                             ResultType->isScalableTy() ?
+                             VecTy->getMinNumElements() * I : I);
+
+          if (ResultType->isScalableTy())
+            ReturnVector =
+              Builder.CreateInsertVector(ResultType, ReturnVector, Ops[I], Idx);
+          else
+            ReturnVector =
+              Builder.CreateIntrinsic(Intrinsic::riscv_vector_insert,
+                                      {ResultType, Ops[I]->getType(), XLenTy},
+                                      {ReturnVector, Ops[I], Idx});
+
         }
         return ReturnVector;
       }
diff --git a/clang/include/clang/Support/RISCVVIntrinsicUtils.h b/clang/include/clang/Support/RISCVVIntrinsicUtils.h
index 97493bae5656e9..33865789041561 100644
--- a/clang/include/clang/Support/RISCVVIntrinsicUtils.h
+++ b/clang/include/clang/Support/RISCVVIntrinsicUtils.h
@@ -429,6 +429,7 @@ class RVVIntrinsic {
   bool hasBuiltinAlias() const { return HasBuiltinAlias; }
   bool hasManualCodegen() const { return !ManualCodegen.empty(); }
   bool isMasked() const { return IsMasked; }
+  llvm::StringRef getOverloadedName() const { return OverloadedName; }
   llvm::StringRef getIRName() const { return IRName; }
   llvm::StringRef getManualCodegen() const { return ManualCodegen; }
   PolicyScheme getPolicyScheme() const { return Scheme; }
diff --git a/clang/lib/CodeGen/CGBuiltin.cpp b/clang/lib/CodeGen/CGBuiltin.cpp
index 5b92f1837980c5..1c7d1f81e9bcc1 100644
--- a/clang/lib/CodeGen/CGBuiltin.cpp
+++ b/clang/lib/CodeGen/CGBuiltin.cpp
@@ -21751,13 +21751,14 @@ Value *CodeGenFunction::EmitRISCVBuiltinExpr(unsigned BuiltinID,
   }
 
   Intrinsic::ID ID = Intrinsic::not_intrinsic;
-  unsigned NF = 1;
   // The 0th bit simulates the `vta` of RVV
   // The 1st bit simulates the `vma` of RVV
   constexpr unsigned RVV_VTA = 0x1;
   constexpr unsigned RVV_VMA = 0x2;
   int PolicyAttrs = 0;
   bool IsMasked = false;
+  // This is used by segment load/store to determine it's llvm type.
+  unsigned SegInstSEW = 8;
 
   // Required for overloaded intrinsics.
   llvm::SmallVector<llvm::Type *, 2> IntrinsicTypes;
diff --git a/clang/lib/CodeGen/CodeGenTypes.cpp b/clang/lib/CodeGen/CodeGenTypes.cpp
index d823c336e39b...
[truncated]

Copy link

github-actions bot commented Jul 8, 2024

✅ With the latest revision this PR passed the C/C++ code formatter.

@4vtomat 4vtomat changed the title model tuple type as llvm type part1 target ext part4 [llvm][RISCV] Support RISCV vector tuple CodeGen and Calling Convention Jul 8, 2024
@4vtomat 4vtomat force-pushed the model_tuple_type_as_llvm_type_part1_target_ext_part4 branch from 290074a to 9d5f092 Compare July 8, 2024 17:23
@4vtomat 4vtomat force-pushed the model_tuple_type_as_llvm_type_part1_target_ext_part4 branch from 9d5f092 to 6fbbabf Compare August 10, 2024 09:40
MVT::getScalableVectorVT(MVT::i8, NumElts),
StoredVal, DAG.getVectorIdxConstant(i, DL));
Ret = DAG.getStore(Chain, DL, Extract, BasePtr,
MachinePointerInfo(Store->getAddressSpace()),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this discarding the pointer info?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The correct pointer info needs a scalable offset, but that doesn't seem to be supported yet. Is there something better we can do?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought that got added?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The offset field in MachinePointerInfo is still an int64_t and getWithOffset takes a int64_t. SelectionDAGBuilder::visitLoad/visitStore still have this code.

    // TODO: MachinePointerInfo only supports a fixed length offset. 
    MachinePointerInfo PtrInfo =                                                 
        !Offsets[i].isScalable() || Offsets[i].isZero()                          
            ? MachinePointerInfo(PtrV, Offsets[i].getKnownMinValue())            
            : MachinePointerInfo();

for (unsigned i = 0; i < NF; ++i) {
SDValue LoadVal = DAG.getLoad(
MVT::getScalableVectorVT(MVT::i8, NumElts), DL, DAG.getEntryNode(),
BasePtr, MachinePointerInfo(Load->getAddressSpace()), Align(8));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this discarding the pointer info?

@4vtomat 4vtomat force-pushed the model_tuple_type_as_llvm_type_part1_target_ext_part4 branch 2 times, most recently from 1c7ba67 to 145083b Compare August 23, 2024 16:03
Copy link
Collaborator

@topperc topperc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

This patch handles target lowering and calling convention.

For target lowering, the vector tuple type represented as multiple
scalable vectors is now changed to a single `MVT`, each `MVT` has a
corresponding register class.

The load/store of vector tuples are handled as the same way but need
another vector insert/extract instructions to get sub-register group.

Inline assembly constraint for vector tuple type can directly be modeled
as "vr" which is identical to normal vector registers.

For calling convention, it no longer needs an alternative algorithm to
handle register allocation, this makes the code easier to maintain and
read.
@4vtomat 4vtomat force-pushed the model_tuple_type_as_llvm_type_part1_target_ext_part4 branch from 262083b to a7f3c0a Compare August 31, 2024 11:27
@4vtomat 4vtomat merged commit 22f9874 into llvm:main Aug 31, 2024
5 of 8 checks passed
@4vtomat 4vtomat deleted the model_tuple_type_as_llvm_type_part1_target_ext_part4 branch August 31, 2024 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend:RISC-V clang:codegen IR generation bugs: mangling, exceptions, etc. clang:frontend Language frontend issues, e.g. anything involving "Sema" clang Clang issues not falling into any other category llvm:analysis llvm:ir llvm:SelectionDAG SelectionDAGISel as well llvm:transforms
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants