Skip to content

Commit 7058633

Browse files
Yonghong Songmehmetb0
Yonghong Song
authored and
mehmetb0
committed
bpf, x64: Fix a jit convergence issue
BugLink: https://bugs.launchpad.net/bugs/2097301 [ Upstream commit c8831bdbfbab672c006a18006d36932a494b2fd6 ] Daniel Hodges reported a jit error when playing with a sched-ext program. The error message is: unexpected jmp_cond padding: -4 bytes But further investigation shows the error is actual due to failed convergence. The following are some analysis: ... pass4, final_proglen=4391: ... 20e: 48 85 ff test rdi,rdi 211: 74 7d je 0x290 213: 48 8b 77 00 mov rsi,QWORD PTR [rdi+0x0] ... 289: 48 85 ff test rdi,rdi 28c: 74 17 je 0x2a5 28e: e9 7f ff ff ff jmp 0x212 293: bf 03 00 00 00 mov edi,0x3 Note that insn at 0x211 is 2-byte cond jump insn for offset 0x7d (-125) and insn at 0x28e is 5-byte jmp insn with offset -129. pass5, final_proglen=4392: ... 20e: 48 85 ff test rdi,rdi 211: 0f 84 80 00 00 00 je 0x297 217: 48 8b 77 00 mov rsi,QWORD PTR [rdi+0x0] ... 28d: 48 85 ff test rdi,rdi 290: 74 1a je 0x2ac 292: eb 84 jmp 0x218 294: bf 03 00 00 00 mov edi,0x3 Note that insn at 0x211 is 6-byte cond jump insn now since its offset becomes 0x80 based on previous round (0x293 - 0x213 = 0x80). At the same time, insn at 0x292 is a 2-byte insn since its offset is -124. pass6 will repeat the same code as in pass4. pass7 will repeat the same code as in pass5, and so on. This will prevent eventual convergence. Passes 1-14 are with padding = 0. At pass15, padding is 1 and related insn looks like: 211: 0f 84 80 00 00 00 je 0x297 217: 48 8b 77 00 mov rsi,QWORD PTR [rdi+0x0] ... 24d: 48 85 d2 test rdx,rdx The similar code in pass14: 211: 74 7d je 0x290 213: 48 8b 77 00 mov rsi,QWORD PTR [rdi+0x0] ... 249: 48 85 d2 test rdx,rdx 24c: 74 21 je 0x26f 24e: 48 01 f7 add rdi,rsi ... Before generating the following insn, 250: 74 21 je 0x273 "padding = 1" enables some checking to ensure nops is either 0 or 4 where #define INSN_SZ_DIFF (((addrs[i] - addrs[i - 1]) - (prog - temp))) nops = INSN_SZ_DIFF - 2 In this specific case, addrs[i] = 0x24e // from pass14 addrs[i-1] = 0x24d // from pass15 prog - temp = 3 // from 'test rdx,rdx' in pass15 so nops = -4 and this triggers the failure. To fix the issue, we need to break cycles of je <-> jmp. For example, in the above case, we have 211: 74 7d je 0x290 the offset is 0x7d. If 2-byte je insn is generated only if the offset is less than 0x7d (<= 0x7c), the cycle can be break and we can achieve the convergence. I did some study on other cases like je <-> je, jmp <-> je and jmp <-> jmp which may cause cycles. Those cases are not from actual reproducible cases since it is pretty hard to construct a test case for them. the results show that the offset <= 0x7b (0x7b = 123) should be enough to cover all cases. This patch added a new helper to generate 8-bit cond/uncond jmp insns only if the offset range is [-128, 123]. Reported-by: Daniel Hodges <hodgesd@meta.com> Signed-off-by: Yonghong Song <yonghong.song@linux.dev> Link: https://lore.kernel.org/r/20240904221251.37109-1-yonghong.song@linux.dev Signed-off-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: Manuel Diewald <manuel.diewald@canonical.com> Signed-off-by: Koichiro Den <koichiro.den@canonical.com>
1 parent 14fc608 commit 7058633

File tree

1 file changed

+52
-2
lines changed

1 file changed

+52
-2
lines changed

‎arch/x86/net/bpf_jit_comp.c

Lines changed: 52 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,56 @@ static bool is_imm8(int value)
6464
return value <= 127 && value >= -128;
6565
}
6666

67+
/*
68+
* Let us limit the positive offset to be <= 123.
69+
* This is to ensure eventual jit convergence For the following patterns:
70+
* ...
71+
* pass4, final_proglen=4391:
72+
* ...
73+
* 20e: 48 85 ff test rdi,rdi
74+
* 211: 74 7d je 0x290
75+
* 213: 48 8b 77 00 mov rsi,QWORD PTR [rdi+0x0]
76+
* ...
77+
* 289: 48 85 ff test rdi,rdi
78+
* 28c: 74 17 je 0x2a5
79+
* 28e: e9 7f ff ff ff jmp 0x212
80+
* 293: bf 03 00 00 00 mov edi,0x3
81+
* Note that insn at 0x211 is 2-byte cond jump insn for offset 0x7d (-125)
82+
* and insn at 0x28e is 5-byte jmp insn with offset -129.
83+
*
84+
* pass5, final_proglen=4392:
85+
* ...
86+
* 20e: 48 85 ff test rdi,rdi
87+
* 211: 0f 84 80 00 00 00 je 0x297
88+
* 217: 48 8b 77 00 mov rsi,QWORD PTR [rdi+0x0]
89+
* ...
90+
* 28d: 48 85 ff test rdi,rdi
91+
* 290: 74 1a je 0x2ac
92+
* 292: eb 84 jmp 0x218
93+
* 294: bf 03 00 00 00 mov edi,0x3
94+
* Note that insn at 0x211 is 6-byte cond jump insn now since its offset
95+
* becomes 0x80 based on previous round (0x293 - 0x213 = 0x80).
96+
* At the same time, insn at 0x292 is a 2-byte insn since its offset is
97+
* -124.
98+
*
99+
* pass6 will repeat the same code as in pass4 and this will prevent
100+
* eventual convergence.
101+
*
102+
* To fix this issue, we need to break je (2->6 bytes) <-> jmp (5->2 bytes)
103+
* cycle in the above. In the above example je offset <= 0x7c should work.
104+
*
105+
* For other cases, je <-> je needs offset <= 0x7b to avoid no convergence
106+
* issue. For jmp <-> je and jmp <-> jmp cases, jmp offset <= 0x7c should
107+
* avoid no convergence issue.
108+
*
109+
* Overall, let us limit the positive offset for 8bit cond/uncond jmp insn
110+
* to maximum 123 (0x7b). This way, the jit pass can eventually converge.
111+
*/
112+
static bool is_imm8_jmp_offset(int value)
113+
{
114+
return value <= 123 && value >= -128;
115+
}
116+
67117
static bool is_simm32(s64 value)
68118
{
69119
return value == (s64)(s32)value;
@@ -1950,7 +2000,7 @@ st: if (is_imm8(insn->off))
19502000
return -EFAULT;
19512001
}
19522002
jmp_offset = addrs[i + insn->off] - addrs[i];
1953-
if (is_imm8(jmp_offset)) {
2003+
if (is_imm8_jmp_offset(jmp_offset)) {
19542004
if (jmp_padding) {
19552005
/* To keep the jmp_offset valid, the extra bytes are
19562006
* padded before the jump insn, so we subtract the
@@ -2032,7 +2082,7 @@ st: if (is_imm8(insn->off))
20322082
break;
20332083
}
20342084
emit_jmp:
2035-
if (is_imm8(jmp_offset)) {
2085+
if (is_imm8_jmp_offset(jmp_offset)) {
20362086
if (jmp_padding) {
20372087
/* To avoid breaking jmp_offset, the extra bytes
20382088
* are padded before the actual jmp insn, so

0 commit comments

Comments
 (0)