Fix -fvk-invert-y

[Nielsbishere]

#7446
This fixes some outdated documentation as well as a compile error when enabling fvk-invert-y on lib files and makes sure that it only gets enabled on SV_POSITION that is used in VS/GS/DS/MS (so PS doesn't get caught in the crossfire).
Also tested the dx-position-w one and that one already has correct behavior here.

[Nielsbishere]

Extra context:

//dxc -T lib_6_3 test.hlsl -spirv -fvk-invert-y

[shader("vertex")]
float4 test(float4 a : A) : SV_POSITION {
    return a;
}

Before fix (can't enable -fvk-invert-y without an error, so would end up like this without the flag):

; SPIR-V
; Version: 1.0
; Generator: Google spiregg; 0
; Bound: 12
; Schema: 0
               OpCapability Shader
               OpMemoryModel Logical GLSL450
               OpEntryPoint Vertex %test "test" %in_var_A %gl_Position
               OpSource HLSL 630
               OpName %in_var_A "in.var.A"
               OpName %test "test"
               OpDecorate %gl_Position BuiltIn Position
               OpDecorate %in_var_A Location 0
      %float = OpTypeFloat 32
    %v4float = OpTypeVector %float 4
%_ptr_Input_v4float = OpTypePointer Input %v4float
%_ptr_Output_v4float = OpTypePointer Output %v4float
       %void = OpTypeVoid
          %9 = OpTypeFunction %void
   %in_var_A = OpVariable %_ptr_Input_v4float Input
%gl_Position = OpVariable %_ptr_Output_v4float Output
       %test = OpFunction %void None %9
         %10 = OpLabel
         %11 = OpLoad %v4float %in_var_A
               OpStore %gl_Position %11
               OpReturn
               OpFunctionEnd

After fix:

; SPIR-V
; Version: 1.0
; Generator: Google spiregg; 0
; Bound: 15
; Schema: 0
               OpCapability Shader
               OpMemoryModel Logical GLSL450
               OpEntryPoint Vertex %test "test" %in_var_A %gl_Position
               OpSource HLSL 630
               OpName %in_var_A "in.var.A"
               OpName %test "test"
               OpDecorate %gl_Position BuiltIn Position
               OpDecorate %in_var_A Location 0
      %float = OpTypeFloat 32
    %v4float = OpTypeVector %float 4
%_ptr_Input_v4float = OpTypePointer Input %v4float
%_ptr_Output_v4float = OpTypePointer Output %v4float
       %void = OpTypeVoid
          %9 = OpTypeFunction %void
   %in_var_A = OpVariable %_ptr_Input_v4float Input
%gl_Position = OpVariable %_ptr_Output_v4float Output
       %test = OpFunction %void None %9
         %10 = OpLabel
         %11 = OpLoad %v4float %in_var_A
         %12 = OpCompositeExtract %float %11 1
         %13 = OpFNegate %float %12
         %14 = OpCompositeInsert %v4float %13 %11 1
               OpStore %gl_Position %14
               OpReturn
               OpFunctionEnd

[llvm-beanz]

We'll need a test case, but the code change looks reasonable to me.

@sperron can you or someone from your team take a look at this?

[s-perron]

I need to look at this carefully. I have some corner cases in mind, and I need to double check if they work.

[s-perron]

Thanks for contributing this is a reasonable implementation. As mentioned above add a test. Include multiple shaders in the test. Try to show one shader inverting y while another does not.

I have one concern about usability, but that is outside of the scope of this PR. You could run into trouble if you have a VS->GS->PS pipeline. If the VS and GS are both compiled with invert-y, then they will both invert it and cancel each other out, right? Just something to watch out for.

[Nielsbishere]

Thanks for contributing this is a reasonable implementation. As mentioned above add a test. Include multiple shaders in the test. Try to show one shader inverting y while another does not.

I have one concern about usability, but that is outside of the scope of this PR. You could run into trouble if you have a VS->GS->PS pipeline. If the VS and GS are both compiled with invert-y, then they will both invert it and cancel each other out, right? Just something to watch out for.

Yes that's actually a big issue, I guess that is an issue already in some pipelines using the old way (like mine) where each shader is compiled with this flag and it actually doesn't make sense in case GS/DS is used. I'm not sure what the right solution would be though; if you only flip VS/MS then you might miss other cases. The only thing I can think of is an annotation on the entrypoint that tells it that you don't want to flip it.
EDIT: Now that I think about it, this can only be solved by the one linking the final version, as VS -> ... -> PS you would want the last stage to flip it and from the compiler you can't know this. But that wouldn't work if lib files are arbitrarily linked, unless the user manually flips these at the right moment.

[github-actions]

✅ With the latest revision this PR passed the C/C++ code formatter.

[Nielsbishere]

After another look, it seems like DS/GS/MS/VS aren't the only ones capable of writing to SV_POSITION. I'm not entirely sure if this means HS is excluded on purpose or by accident. But I made it behave as before.