[x86 disassembler] L bit in VEX prefix is not ignored properly

[llvmbot]

Bugzilla Link	10676
Resolution	FIXED
Resolved on	Nov 07, 2018 00:17
Version	trunk
OS	All
Blocks	llvm/llvm-bugzilla-archive#10988
Reporter	LLVM Bugzilla Contributor
CC	@topperc

Extended Description

According to the "Intel® 64 and IA-32 Architectures Software Developer’s Manual
Volume 2 (2A & 2B): Instruction Set Reference, A-Z", p. 3-42:

ADDSD—Add Scalar Double-Precision Floating-Point Values
VEX.NDS.LIG.F2.0F.WIG 58 /r

"LIG" in the above spec is defined on p. 3-5:
If VEX.LIG is present in the opcode column: The VEX.L value is ignored.

With llvm-mc trunk revision 135913:
$ echo '0xc5 0xf3 0x58 0xc0'| ./llvm-mc -disassemble -triple="x86_64"
vaddsd %xmm0, %xmm1, %xmm0
$ echo '0xc5 0xf7 0x58 0xc0'| ./llvm-mc -disassemble -triple="x86_64"
vaddps %ymm0, %ymm1, %ymm0

Setting the 'L' bit changed the disassembly from vaddsd to vaddps, but it shouldn't have affected anything according to the docs.

Using the 3-byte VEX prefix has the same behavior:
$ echo '0xc4 0xc1 0x73 0x58 0xc0'| ./llvm-mc -disassemble -triple="x86_64"
vaddsd %xmm8, %xmm1, %xmm0
$ echo '0xc4 0xc1 0x77 0x58 0xc0'| ./llvm-mc -disassemble -triple="x86_64"
vaddps %ymm8, %ymm1, %ymm0

[llvmbot]

vdivsd has a similar problem:

$ echo '0xc5 0xfb 0x5e 0xc0'| ./llvm-mc -disassemble -triple="x86_64" -x86-asm-syntax=intel
vdivsd XMM0, XMM0, XMM0

$ echo '0xc5 0xff 0x5e 0xc0'| ./llvm-mc -disassemble -triple="x86_64" -x86-asm-syntax=intel
vdivps YMM0, YMM0, YMM0

[llvmbot]

vdivss is also similar:
$ echo '0xc5 0xfa 0x5e 0xc0'| ./llvm-mc -disassemble -triple="x86_64" -x86-asm-syntax=intel
vdivss XMM0, XMM0, XMM0
$ echo '0xc5 0xfe 0x5e 0xc0'| ./llvm-mc -disassemble -triple="x86_64" -x86-asm-syntax=intel
vdivps YMM0, YMM0, YMM0

[llvmbot]

The FP compare insts are mentioned in bug 10678, but I'm listing them here since it appears to be another case of not ignoring the L bit as expected:

$ echo '0xc5 0xf3 0xc2 0xc5 0x01'| ./llvm-mc -disassemble -triple="x86_64"
vcmpsd $1, %xmm5, %xmm1, %xmm0
$ echo '0xc4 0xc1 0x73 0xc2 0xc5 0x01'| ./llvm-mc -disassemble -triple="x86_64"
vcmpsd $1, %xmm13, %xmm1, %xmm0

And then set the L bit for either VEX prefix form:

$ echo '0xc5 0xf7 0xc2 0xc5 0x01'| ./llvm-mc -disassemble -triple="x86_64"
:1:1: warning: invalid instruction encoding
0xc5 0xf7 0xc2 0xc5 0x01
^
:1:16: warning: invalid instruction encoding
0xc5 0xf7 0xc2 0xc5 0x01
^
$ echo '0xc4 0xc1 0x77 0xc2 0xc5 0x01'| ./llvm-mc -disassemble -triple="x86_64"
:1:1: warning: invalid instruction encoding
0xc4 0xc1 0x77 0xc2 0xc5 0x01
^
:1:21: warning: invalid instruction encoding
0xc4 0xc1 0x77 0xc2 0xc5 0x01

[llvmbot]

vmovsd has the same problem:
$ echo '0xc5 0xf3 0x10 0xc0'| ./llvm-mc -disassemble -triple="x86_64" -x86-asm-syntax=intel
vmovsd XMM0, XMM1, XMM0
$ echo '0xc5 0xf7 0x10 0xc0'| ./llvm-mc -disassemble -triple="x86_64" -x86-asm-syntax=intel
:1:1: warning: invalid instruction encoding
0xc5 0xf7 0x10 0xc0

[llvmbot]

$ echo '0xc5 0xf2 0x10 0xc0'| ./llvm-mc -disassemble -triple="x86_64"
vmovss %xmm0, %xmm1, %xmm0

$ echo '0xc5 0xf6 0x10 0xc0'| ./llvm-mc -disassemble -triple="x86_64"
:1:1: warning: invalid instruction encoding
0xc5 0xf6 0x10 0xc0
^

[llvmbot]

Another case of 'L' bit not ignored - this was originally noted in bug 10724:
$ echo '0xc5 0xfb 0x2d 0xc0'| ./llvm-mc -disassemble -triple="x86_64" -x86-asm-syntax=intel
vcvtsd2si EAX, XMM0
$ echo '0xc5 0xff 0x2d 0xc0'| ./llvm-mc -disassemble -triple="x86_64" -x86-asm-syntax=intel
:1:1: warning: invalid instruction encoding
0xc5 0xff 0x2d 0xc0
^
:1:16: warning: invalid instruction encoding
0xc5 0xff 0x2d 0xc0
^
$ echo '0xc4 0xe1 0xfb 0x2d 0xc0'| ./llvm-mc -disassemble -triple="x86_64" -x86-asm-syntax=intel
vcvtsd2si RAX, XMM0
$ echo '0xc4 0xe1 0xff 0x2d 0xc0'| ./llvm-mc -disassemble -triple="x86_64" -x86-asm-syntax=intel
:1:1: warning: invalid instruction encoding
0xc4 0xe1 0xff 0x2d 0xc0
^
:1:21: warning: invalid instruction encoding
0xc4 0xe1 0xff 0x2d 0xc0
^

[llvmbot]

More FP compare test cases from bug 10678:
0xc5 0xf9 0x2f 0xc1 #vcomisd
0xc4 0xc1 0x79 0x2f 0xc1 #vcomisd

0xc5 0xf8 0x2f 0xc1 #vcomiss
0xc4 0xc1 0x78 0x2f 0xc1 #vcomiss

Those work since the 'L' bit isn't set. But these don't (using r139612):
0xc5 0xfb 0x2f 0xc1 #vcomisd
0xc4 0xc1 0x7b 0x2f 0xc1 #vcomisd

0xc5 0xfc 0x2f 0xc1 #vcomiss
0xc4 0xc1 0x7c 0x2f 0xc1 #vcomiss

[topperc]

Fixed in 141065.

[llvmbot]

Using llvm-mc built from r141535, all of the cases listed in this bug report work, except the vcomisd tests shown in comment 7:

$ echo '0xc4 0xc1 0x7b 0x2f 0xc1'| ./llvm-mc -disassemble -triple="x86_64"
:1:1: warning: invalid instruction encoding
0xc4 0xc1 0x7b 0x2f 0xc1
^
:1:21: warning: invalid instruction encoding
0xc4 0xc1 0x7b 0x2f 0xc1

$ echo '0xc5 0xfb 0x2f 0xc1'| ./llvm-mc -disassemble -triple="x86_64"
:1:1: warning: invalid instruction encoding
0xc5 0xfb 0x2f 0xc1
^
:1:16: warning: invalid instruction encoding
0xc5 0xfb 0x2f 0xc1
^

[topperc]

The L bit is bit 2 and 0xb in binary is 1011 which doesn't have bit 2 set. Perhaps you wanted 0xd?

Can you try
0xc4 0xc1 0x7d 0x2f 0xc1

[llvmbot]

Oops! Yes, my test case was off.

All of the examples here are working now. Thanks!

[bcardosolopes]

mentioned in issue llvm/llvm-bugzilla-archive#10988