[AVX-512] Fused-load doesn't work for `<8 x i64>{ (PowerOf2 - 1), 0, 0, 0, 0, 0, 0, 0 }`

[Validark]

Godbolt

define dso_local zeroext i1 @greaterThan1(<8 x i64> %0) local_unnamed_addr {
Entry:
  %1 = icmp ugt <8 x i64> %0, <i64 1, i64 0, i64 0, i64 0, i64 0, i64 0, i64 0, i64 0>
  %2 = bitcast <8 x i1> %1 to i8
  %3 = icmp ne i8 %2, 0
  ret i1 %3
}

Compiled with -O3 -march=znver5:

.LCPI0_1:
        .byte   1
        .byte   0
greaterThan1:
.LgreaterThan1$local:
        vpmovsxbq       xmm1, word ptr [rip + .LCPI0_1]
        vpcmpnleuq      k0, zmm0, zmm1
        kortestb        k0, k0
        setne   al
        vzeroupper
        ret

Should be:

.LCPI0_0:
        .quad   1
        .quad   0
        .quad   0
        .quad   0
        .quad   0
        .quad   0
        .quad   0
        .quad   0
greaterThan1:
.LgreaterThan1$local:
        vpcmpnleuq      k0, zmm0, zmmword ptr [rip + .LCPI0_0]
        kortestb        k0, k0
        setne   al
        vzeroupper
        ret

[llvmbot]

@llvm/issue-subscribers-backend-x86

Author: Niles Salter (Validark)

[Godbolt](https://llvm.godbolt.org/#g:!((g:!((g:!((h:codeEditor,i:(filename:'1',fontScale:14,fontUsePx:'0',j:1,lang:llvm,selection:(endColumn:85,endLineNumber:3,positionColumn:85,positionLineNumber:3,selectionStartColumn:85,selectionStartLineNumber:3,startColumn:85,startLineNumber:3),source:'define+dso_local+zeroext+i1+@greaterThan1(%3C8+x+i64%3E+%250)+local_unnamed_addr+%7B%0AEntry:%0A++%251+%3D+icmp+ugt+%3C8+x+i64%3E+%250,+%3Ci64+1,+i64+0,+i64+0,+i64+0,+i64+0,+i64+0,+i64+0,+i64+0%3E%0A++%252+%3D+bitcast+%3C8+x+i1%3E+%251+to+i8%0A++%253+%3D+icmp+ne+i8+%252,+0%0A++ret+i1+%253%0A%7D%0A'),l:'5',n:'0',o:'LLVM+IR+source+%231',t:'0')),k:49.178082191780824,l:'4',n:'0',o:'',s:0,t:'0'),(g:!((h:compiler,i:(compiler:irclangtrunk,filters:(b:'0',binary:'1',binaryObject:'1',commentOnly:'0',debugCalls:'1',demangle:'0',directives:'0',execute:'1',intel:'0',libraryCode:'0',trim:'1',verboseDemangling:'0'),flagsViewOpen:'1',fontScale:14,fontUsePx:'0',j:1,lang:llvm,libs:!(),options:'-O3+-march%3Dznver5',overrides:!(),selection:(endColumn:1,endLineNumber:1,positionColumn:1,positionLineNumber:1,selectionStartColumn:1,selectionStartLineNumber:1,startColumn:1,startLineNumber:1),source:1),l:'5',n:'0',o:'+clang+(trunk)+(Editor+%231)',t:'0')),k:50.82191780821918,l:'4',m:100,n:'0',o:'',s:0,t:'0')),l:'2',n:'0',o:'',t:'0')),version:4)

define dso_local zeroext i1 @<!-- -->greaterThan1(&lt;8 x i64&gt; %0) local_unnamed_addr {
Entry:
  %1 = icmp ugt &lt;8 x i64&gt; %0, &lt;i64 1, i64 0, i64 0, i64 0, i64 0, i64 0, i64 0, i64 0&gt;
  %2 = bitcast &lt;8 x i1&gt; %1 to i8
  %3 = icmp ne i8 %2, 0
  ret i1 %3
}

Compiled with -O3 -march=znver5:

.LCPI0_1:
        .byte   1
        .byte   0
greaterThan1:
.LgreaterThan1$local:
        vpmovsxbq       xmm1, word ptr [rip + .LCPI0_1]
        vpcmpnleuq      k0, zmm0, zmm1
        kortestb        k0, k0
        setne   al
        vzeroupper
        ret

Should be:

.LCPI0_0:
        .quad   1
        .quad   0
        .quad   0
        .quad   0
        .quad   0
        .quad   0
        .quad   0
        .quad   0
greaterThan1:
.LgreaterThan1$local:
        vpcmpnleuq      k0, zmm0, zmmword ptr [rip + .LCPI0_0]
        kortestb        k0, k0
        setne   al
        vzeroupper
        ret

[Validark]

Seems the same thing happens for 1, 3, 7, 15, etc. Updated title.

[phoebewang]

I think it's caused by X86FixupVectorConstantsPass too. Compiler reduced the load size from 512 to 128. It's hard to judge which one is better without benchmark data.

[Validark]

I think it's caused by X86FixupVectorConstantsPass too. Compiler reduced the load size from 512 to 128. It's hard to judge which one is better without benchmark data.

The compiler prefers the fused-load version for speed-optimized builds in virtually all other cases. For me, that's enough to say it should be fused, unless the vector needs to be used more than once. It's weird that this one case randomly has a different behavior.

[RKSimon]

No, X86FixupVectorConstantsPass doesn't unfold loads - its just replaced a 128-bit VMOVDQA with VPMOVSXBQ to save constant pool space on a independent load instruction.

BUILDVECTOR lowering decided to replace the 512-bit constant (with all upper zeros) with:

Optimized legalized selection DAG: %bb.0 'greaterThan1:Entry'
SelectionDAG has 21 nodes:
  t0: ch,glue = EntryToken
    t2: v8i64,ch = CopyFromReg t0, Register:v8i64 %0
        t37: v16i32 = BUILD_VECTOR Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>, Constant:i32<0>
      t38: v8i64 = bitcast t37
        t48: i64 = X86ISD::Wrapper TargetConstantPool:i64<<4 x i32> <i32 1, i32 0, i32 0, i32 0>> 0
      t49: v2i64,ch = load<(load (s128) from constant-pool)> t0, t48, undef:i64
    t39: v8i64 = insert_subvector t38, t49, Constant:i64<0>
  t7: v8i1 = setcc t2, t39, setugt:ch
      t23: i32 = X86ISD::KORTEST t7, t7
    t24: i8 = X86ISD::SETCC TargetConstant:i8<5>, t23
  t15: ch,glue = CopyToReg t0, Register:i8 $al, t24
  t16: ch = X86ISD::RET_GLUE t15, TargetConstant:i32<0>, Register:i8 $al, t15:1