Qiling fails to emulate simple binary with a ubuntu-22.04 root fs

[liona24]

*Describe the bug
When trying to emulate simple binaries on a vanilla ubuntu 22.04 qiling fails gloriously with unicorn.unicorn.UcError: Invalid memory fetch (UC_ERR_FETCH_UNMAPPED)

Sample Code

FROM ubuntu:22.04

RUN apt-get update --fix-missing
RUN apt-get install -y --no-install-recommends python3-minimal python3-pip build-essential

RUN pip install qiling

RUN echo '#include <stdio.h>\nint main() { puts("Hi"); }' > main.c && \
    gcc main.c -o main

ENTRYPOINT [ "qltool", "run", "--filename", "main", "--rootfs", "/" ]

Expected behavior
The emulations is successful, binary prints "Hi". This works fine on f.e. ubuntu 20.04

Additional context
This is the entire output log, but you should be able to reproduce with the given dockerfile:

[=]     brk(inp = 0x0) = 0x55555555b000
[!]     prctl code 0x3001 not implemented
[=]     arch_prctl(code = 0x3001, addr = 0x80000000dcc0) = 0x0
[=]     uname(buf = 0x80000000d8a0) = 0x0
[=]     access(path = 0x7ffff7e04d90, mode = 0x4) = -0x1 (EPERM)
[=]     openat(fd = 0xffffff9c, path = 0x7ffff7e0321b, flags = 0x80000, mode = 0x0) = 0x3
[=]     newfstatat(dirfd = 0x3, path = 0x7ffff7e03ee9, buf_ptr = 0x80000000ce10, flags = 0x1000) = 0x0
[=]     mmap(addr = 0x0, length = 0x1e89, prot = 0x1, flags = 0x2, fd = 0x3, pgoffset = 0x0) = 0x7fffb7dd6000
[=]     close(fd = 0x3) = 0x0
[=]     openat(fd = 0xffffff9c, path = 0x7ffff7e10aa0, flags = 0x80000, mode = 0x0) = 0x3
[=]     read(fd = 0x3, buf = 0x80000000d048, length = 0x340) = 0x340
[=]     pread64(fd = 0x3, buf = 0x80000000cc50, length = 0x310, offt = 0x40) = 0x310
[=]     pread64(fd = 0x3, buf = 0x80000000cc10, length = 0x30, offt = 0x350) = 0x30
[=]     pread64(fd = 0x3, buf = 0x80000000cbc0, length = 0x44, offt = 0x380) = 0x44
[=]     newfstatat(dirfd = 0x3, path = 0x7ffff7e03ee9, buf_ptr = 0x80000000cee0, flags = 0x1000) = 0x0
[=]     pread64(fd = 0x3, buf = 0x80000000cb20, length = 0x310, offt = 0x40) = 0x310
[=]     mmap(addr = 0x0, length = 0x227e50, prot = 0x1, flags = 0x802, fd = 0x3, pgoffset = 0x0) = 0x7fffb7dd8000
[=]     mmap(addr = 0x7fffb7e00000, length = 0x195000, prot = 0x5, flags = 0x812, fd = 0x3, pgoffset = 0x28000) = 0x7fffb7e00000
[=]     mmap(addr = 0x7fffb7f95000, length = 0x58000, prot = 0x1, flags = 0x812, fd = 0x3, pgoffset = 0x1bd000) = 0x7fffb7f95000
[=]     mmap(addr = 0x7fffb7fed000, length = 0x6000, prot = 0x3, flags = 0x812, fd = 0x3, pgoffset = 0x214000) = 0x7fffb7fed000
[=]     mmap(addr = 0x7fffb7ff3000, length = 0xce50, prot = 0x3, flags = 0x32, fd = 0xffffffff, pgoffset = 0x0) = 0x7fffb7ff3000
[=]     close(fd = 0x3) = 0x0
[=]     mmap(addr = 0x0, length = 0x2000, prot = 0x3, flags = 0x22, fd = 0xffffffff, pgoffset = 0x0) = 0x7fffb8000000
[=]     arch_prctl(code = 0x1002, addr = 0x7fffb80010c0) = 0x0
[=]     set_tid_address(tidptr = 0x7fffb8001390) = 0x1
[=]     set_robust_list(head_ptr = 0x7fffb80013a0, head_len = 0x18) = 0x0
[!]     0x7ffff7dea1cf: syscall ql_syscall_rseq number = 0x14e(334) not implemented
/lib/x86_64-linux-gnu/libc.so.6: CPU ISA level is lower than required
[=]     writev(fd = 0x2, vec = 0x80000000d530, vlen = 0x2) = 0x46
[=]     exit_group(code = 0x7f) = ?
[x]     CPU Context:
[x]     ah      : 0x0
[x]     al      : 0xe7
[x]     ch      : 0x0
[x]     cl      : 0x0
[x]     dh      : 0xd9
[x]     dl      : 0x70
[x]     bh      : 0x0
[x]     bl      : 0x0
[x]     ax      : 0xe7
[x]     cx      : 0x0
[x]     dx      : 0xd970
[x]     bx      : 0x0
[x]     sp      : 0xd958
[x]     bp      : 0x0
[x]     si      : 0x4fc9
[x]     di      : 0x5149
[x]     ip      : 0x0
[x]     eax     : 0xe7
[x]     ecx     : 0x0
[x]     edx     : 0xd970
[x]     ebx     : 0x0
[x]     esp     : 0xd958
[x]     ebp     : 0x0
[x]     esi     : 0xf7de4fc9
[x]     edi     : 0x55555149
[x]     eip     : 0x0
[x]     rax     : 0xe7
[x]     rbx     : 0x0
[x]     rcx     : 0x0
[x]     rdx     : 0x80000000d970
[x]     rsi     : 0x7ffff7de4fc9
[x]     rdi     : 0x555555555149
[x]     rbp     : 0x0
[x]     rsp     : 0x80000000d958
[x]     r8      : 0x0
[x]     r9      : 0x3c
[x]     r10     : 0xffffffff
[x]     r11     : 0x90
[x]     r12     : 0x0
[x]     r13     : 0x0
[x]     r14     : 0x7fffb80010c0
[x]     r15     : 0x7ffff7e102e0
[x]     rip     : 0x0
[x]     cr0     : 0x11
[x]     cr1     : 0x0
[x]     cr2     : 0x0
[x]     cr3     : 0x0
[x]     cr4     : 0x0
[x]     cr8     : 0x0
[x]     st0     : 0x0
[x]     st1     : 0x0
[x]     st2     : 0x0
[x]     st3     : 0x0
[x]     st4     : 0x0
[x]     st5     : 0x0
[x]     st6     : 0x0
[x]     st7     : 0x0
[x]     eflags  : 0x46
[x]     cs      : 0x33
[x]     ss      : 0x28
[x]     ds      : 0x0
[x]     es      : 0x0
[x]     fs      : 0x0
[x]     gs      : 0x0
[x]     r8b     : 0x0
[x]     r9b     : 0x3c
[x]     r10b    : 0xff
[x]     r11b    : 0x90
[x]     r12b    : 0x0
[x]     r13b    : 0x0
[x]     r14b    : 0xc0
[x]     r15b    : 0xe0
[x]     r8w     : 0x0
[x]     r9w     : 0x3c
[x]     r10w    : 0xffff
[x]     r11w    : 0x90
[x]     r12w    : 0x0
[x]     r13w    : 0x0
[x]     r14w    : 0x10c0
[x]     r15w    : 0x2e0
[x]     r8d     : 0x0
[x]     r9d     : 0x3c
[x]     r10d    : 0xffffffff
[x]     r11d    : 0x90
[x]     r12d    : 0x0
[x]     r13d    : 0x0
[x]     r14d    : 0xb80010c0
[x]     r15d    : 0xf7e102e0
[x]     fsbase  : 0x7fffb80010c0
[x]     gsbase  : 0x6000000
[x]     xmm0    : 0x0
[x]     xmm1    : 0xff000000000000000000000000000000
[x]     xmm2    : 0xffff000000000000ffff0000000000ff
[x]     xmm3    : 0xffff000000000000ffff000000000000
[x]     xmm4    : 0x0
[x]     xmm5    : 0x2815000000000000004b
[x]     xmm6    : 0x2815000000000000004b
[x]     xmm7    : 0x1
[x]     xmm8    : 0x0
[x]     xmm9    : 0x0
[x]     xmm10   : 0x0
[x]     xmm11   : 0x0
[x]     xmm12   : 0x0
[x]     xmm13   : 0x0
[x]     xmm14   : 0x0
[x]     xmm15   : 0x0
[x]     xmm16   : 0x0
[x]     xmm17   : 0x0
[x]     xmm18   : 0x0
[x]     xmm19   : 0x0
[x]     xmm20   : 0x0
[x]     xmm21   : 0x0
[x]     xmm22   : 0x0
[x]     xmm23   : 0x0
[x]     xmm24   : 0x0
[x]     xmm25   : 0x0
[x]     xmm26   : 0x0
[x]     xmm27   : 0x0
[x]     xmm28   : 0x0
[x]     xmm29   : 0x0
[x]     xmm30   : 0x0
[x]     xmm31   : 0x0
[x]     ymm0    : 0x0
[x]     ymm1    : 0xff000000000000000000000000000000
[x]     ymm2    : 0xffff000000000000ffff0000000000ff
[x]     ymm3    : 0xffff000000000000ffff000000000000
[x]     ymm4    : 0x0
[x]     ymm5    : 0x2815000000000000004b
[x]     ymm6    : 0x2815000000000000004b
[x]     ymm7    : 0x1
[x]     ymm8    : 0x0
[x]     ymm9    : 0x0
[x]     ymm10   : 0x0
[x]     ymm11   : 0x0
[x]     ymm12   : 0x0
[x]     ymm13   : 0x0
[x]     ymm14   : 0x0
[x]     ymm15   : 0x0
[x]     ymm16   : 0x0
[x]     ymm17   : 0x0
[x]     ymm18   : 0x0
[x]     ymm19   : 0x0
[x]     ymm20   : 0x0
[x]     ymm21   : 0x0
[x]     ymm22   : 0x0
[x]     ymm23   : 0x0
[x]     ymm24   : 0x0
[x]     ymm25   : 0x0
[x]     ymm26   : 0x0
[x]     ymm27   : 0x0
[x]     ymm28   : 0x0
[x]     ymm29   : 0x0
[x]     ymm30   : 0x0
[x]     ymm31   : 0x0
[x]     zmm0    : 0x0
[x]     zmm1    : 0x0
[x]     zmm2    : 0x0
[x]     zmm3    : 0x0
[x]     zmm4    : 0x0
[x]     zmm5    : 0x0
[x]     zmm6    : 0x0
[x]     zmm7    : 0x0
[x]     zmm8    : 0x0
[x]     zmm9    : 0x0
[x]     zmm10   : 0x0
[x]     zmm11   : 0x0
[x]     zmm12   : 0x0
[x]     zmm13   : 0x0
[x]     zmm14   : 0x0
[x]     zmm15   : 0x0
[x]     zmm16   : 0x0
[x]     zmm17   : 0x0
[x]     zmm18   : 0x0
[x]     zmm19   : 0x0
[x]     zmm20   : 0x0
[x]     zmm21   : 0x0
[x]     zmm22   : 0x0
[x]     zmm23   : 0x0
[x]     zmm24   : 0x0
[x]     zmm25   : 0x0
[x]     zmm26   : 0x0
[x]     zmm27   : 0x0
[x]     zmm28   : 0x0
[x]     zmm29   : 0x0
[x]     zmm30   : 0x0
[x]     zmm31   : 0x0
[x]     PC = 0x0000000000000000 (unreachable)

[x]     Memory map:
[x]     Start                End                  Perm    Label                  Image
[x]     000000000000030000 - 000000000000031000   rwx     [GDT]                  
[x]     000000555555554000 - 000000555555555000   r--     main                   /main
[x]     000000555555555000 - 000000555555556000   r-x     main                   /main
[x]     000000555555556000 - 000000555555557000   r--     main                   /main
[x]     000000555555557000 - 000000555555559000   rw-     main                   /main
[x]     000000555555559000 - 00000055555555b000   rwx     [hook_mem]             
[x]     0000007fffb7dd6000 - 0000007fffb7dd8000   r--     [mmap] ld.so.cache     
[x]     0000007fffb7dd8000 - 0000007fffb7e00000   r--     [mmap] libc.so.6       
[x]     0000007fffb7e00000 - 0000007fffb7f95000   r-x     [mmap] libc.so.6       
[x]     0000007fffb7f95000 - 0000007fffb7fed000   r--     [mmap] libc.so.6       
[x]     0000007fffb7fed000 - 0000007fffb7ff3000   rw-     [mmap] libc.so.6       
[x]     0000007fffb7ff3000 - 0000007fffb8000000   rw-     [mmap] libc.so.6       
[x]     0000007fffb8000000 - 0000007fffb8002000   rw-     [syscall_mmap]         
[x]     0000007ffff7dd5000 - 0000007ffff7dd7000   r--     ld-linux-x86-64.so.2   //lib64/ld-linux-x86-64.so.2
[x]     0000007ffff7dd7000 - 0000007ffff7e01000   r-x     ld-linux-x86-64.so.2   //lib64/ld-linux-x86-64.so.2
[x]     0000007ffff7e01000 - 0000007ffff7e0c000   r--     ld-linux-x86-64.so.2   //lib64/ld-linux-x86-64.so.2
[x]     0000007ffff7e0d000 - 0000007ffff7e11000   rw-     ld-linux-x86-64.so.2   //lib64/ld-linux-x86-64.so.2
[x]     0000007ffffffde000 - 00000080000000e000   rwx     [stack]                
[x]     00ffffffffff600000 - 00ffffffffff601000   rwx     [vsyscall]             
Traceback (most recent call last):
  File "/usr/local/bin/qltool", line 253, in <module>
    ql.run(timeout=options.timeout)
  File "/usr/local/lib/python3.10/dist-packages/qiling/core.py", line 573, in run
    self.os.run()
  File "/usr/local/lib/python3.10/dist-packages/qiling/os/linux/linux.py", line 163, in run
    self.ql.emu_start(self.ql.loader.elf_entry, self.exit_point, self.ql.timeout, self.ql.count)
  File "/usr/local/lib/python3.10/dist-packages/qiling/core.py", line 706, in emu_start
    self.uc.emu_start(begin, end, timeout, count)
  File "/usr/local/lib/python3.10/dist-packages/unicorn/unicorn.py", line 548, in emu_start
    raise UcError(status)
unicorn.unicorn.UcError: Invalid memory fetch (UC_ERR_FETCH_UNMAPPED)

[elicn]

Hi there.
It looks like your binary fails to run because of the following error message:

/lib/x86_64-linux-gnu/libc.so.6: CPU ISA level is lower than required

That said, we'll look into why Qiling did not fail gracefully.

[liona24]

May this be a direct cause of the preceding syscall failure?

0x7ffff7dea1cf: syscall ql_syscall_rseq number = 0x14e(334) not implemented

[elicn]

That might be the problem.
Can you try again but with the following code added to your script? (somewhere between ql initialization and ql.run)

def null_rseq_impl(ql: Qiling, abi: int, length: int, flags: int, sig: int):
    # do nothing
    pass

ql.os.set_syscall('rseq', null_rseq_impl, QL_INTERCEPT.CALL)

Notes:

• If the specifying 'rseq' doesn't work, use its numeric id instead: 334
• Not sure what this syscall is expected to return, so you could try replacing pass with return 1 or return 0

[liona24]

I tried that, though it did not help. Neither combination of return values was successful.

[c2r1w]

im also facing same problem did you find any workaround

[liona24]

@LoverOfSadness just use an older version of libc. If you really need the newer version, the easiest way is probably patching the ISA checks in libc

[c2r1w]

Ok :-)

[elicn]

@xwings consider adding an Ubuntu 22.04 LTS machine to the test suite.

[yikesoftware]

I found a temporary solution, which is to skip this error through hook.

hook code:

...
def by_pass_isa_check(ql: Qiling) -> None:
    print("by_pass_isa_check():")
    ql.arch.regs.rip += 0x15
    pass
...
ld_so_base = 0x7ffff7dd5000
ql.hook_address(by_pass_isa_check, ld_so_base+0x2389f) # ld.so base + jz offset

Source:

IDA Disassembly:

[yikesoftware]

which glibc version u using

Glibc2.35 in ubuntu 22.04.

I'm not sure whether this offset is appropriate for your version, but the method is the same.

[elicn]

Some background for this:
glibc attempts to determine whether the underlying system supports certain ISA capabilities the program requires. This is done by executing the cpuid instruction using different leafs, and testing for the supported "ISA Level". There are 4 incremental ISA levels: baseline, v2, v3 and v4, where each one represents a set of CPU features / intructions. Supporting an ISA level means all the CPU features included in that level and previous ones are supported.

Emulating a simple cpuid intruction with Unicorn 2.0 shows that neither of the ISA levels are fully supported, which is probably why glib's test is failing. Here is a table that summarizes what is actually supported by Unicorn 2.0:

References:

• cpu-features.c
• get-isa-level.h
• cpu-features.h

cc: @wtdcode, in case you want to patch Unicorn to support the mising CPU features (or they are already supported, but that support is not reflected in cpuid output)

[wtdcode]

Unicorn has a special hook for cpuid: https://github.com/unicorn-engine/unicorn/blob/6c1cbef6ac505d355033aef1176b684d02e1eb3a/include/unicorn/x86.h#L86

Another solution is that the CPU model might be changed by uc_ctl_set_cpu_model however Qiling doesn't offer this feature so far.

[elicn]

I'll work on enabling a CPU selection.
Hooking the CPUID may work around this in the short-term, but not really a fix.