SM6.8 Barrier ops: not fully validated for stage or correctly represent constraints for RDAT

[tex3d]

Description
New barrier operations in SM 6.8 are not validated for cases only allowed in certain shader models.

The new Barrier operation is underspecified, and has some problems.

• issues with the definitions of equivalent new barrier operations for old barrier intrinsics:
  • it uses ALL_MEMORY where the old intrinsic would not have had any concept of node record memory
  • DeviceMemoryBarrier* intrinsics should exclude GROUP_SHARED_MEMORY
• issues with DXIL definitions:
  • only one incorrect definition for intrinsic receiving handle (mixes name for UAV intrinsic with node record handle type).
  • it's not defined what we do with old intrinsics for SM 6.8: map to new? keep old?
  • it's not defined whether you can use new intrinsics to define equivalent barriers to old intrinsics on lower shader models.
• overly broad/loose definitions
  • Missing rules to constrain legal uses to certain argument combinations and contexts that make sense.

Incoming proposed spec updates:

• Uniform control flow only required for GROUP_SYNC. DeviceMemoryBarrier*() excludes GROUP_SHARED_MEMORY.
• MemoryTypeFlags masked for shader stage, allowing you to use ALL_MEMORY when some memory types are not applicable.
• More detailed rules for valid Barrier() use.
• Fix DXIL ops for UAV Handle vs. NodeRecordHandle.
• Old intrinsics should produce the new DXIL op on SM 6.8.
• Uses of the new HLSL intrinsic equivalent to the old HLSL intrinsics will map to the existing barrier DXIL op on prior shader models.
• A set of flag combinations for translation between SM 6.8 and the older barrier DXIL operation

DXC Sub-Tasks

• Barrier ops: fix legal models and stages for RDAT #6266
• DxilValidation: Stricter validation for new DXIL barrier ops. #6292
• SemaHLSL: Detect invalid uses of new Barrier intrinsic. #6293

Missing Validation Example
Expect validation errors when using Barrier with GROUP_SYNC or GROUP_SCOPE when shader stage does not have access to that scope.

Example test case:

; RUN: %dxilver 1.8 | %dxv %s | FileCheck %s

; CHECK-DAG: Function: main: error: sync in a non-Compute/Amplification/Mesh/Node Shader must only sync UAV (sync_uglobal).
; CHECK-DAG: note: at 'call void @dx.op.barrierByMemoryType(i32 244, i32 1, i32 3)' in block '#0' of function 'main'.

target datalayout = "e-m:e-p:32:32-i1:32-i8:32-i16:32-i32:32-i64:64-f16:32-f32:32-f64:64-n8:16:32:64"
target triple = "dxil-ms-dx"

define void @main() {
  ; MemoryTypeFlags = UAV_MEMORY
  ; SemanticFlags = GROUP_SYNC | GROUP_SCOPE
  call void @dx.op.barrierByMemoryType(i32 244, i32 1, i32 3)  ; BarrierByMemoryType(MemoryTypeFlags,SemanticFlags)
  ret void
}

; Function Attrs: nounwind
declare void @dx.op.storeOutput.f32(i32, i32, i32, i8, float) #0

; Function Attrs: noduplicate nounwind
declare void @dx.op.barrierByMemoryType(i32, i32, i32) #1

attributes #0 = { nounwind }
attributes #1 = { noduplicate nounwind }

!llvm.ident = !{!0}
!dx.version = !{!1}
!dx.valver = !{!1}
!dx.shaderModel = !{!2}
!dx.viewIdState = !{!3}
!dx.entryPoints = !{!4}

!0 = !{!"dxc(private) 1.7.0.4429 (rdat-minsm-check-flags, 9d3b6ba57)"}
!1 = !{i32 1, i32 8}
!2 = !{!"ps", i32 6, i32 8}
!3 = !{[2 x i32] [i32 0, i32 4]}
!4 = !{void ()* @main, !"main", !5, null, null}
!5 = !{null, !6, null}
!6 = !{!7}
!7 = !{i32 0, !"SV_Target", i8 9, i8 16, !8, i8 0, i32 1, i8 4, i32 0, i8 0, !9}
!8 = !{i32 0}
!9 = !{i32 3, i32 15}

Actual Behavior
DxilValidation passes, even though the semantic flags require a compute-like target here.

Environment

• DXC version: recent main branch
• Host Operating System: any