Repository navigation
Usage of libv4l causes a crash during rr record #2929
Description
georgewsinger
Issue body actions
Reactions are currently unavailable
Activity
That looks like VIDIOC_G_INPUT.
Try again with the latest tip?
We now get
[FATAL /build/source/src/RecordSession.cc:345:handle_seccomp_traced_syscall()] (task 78878 (rec:78878) at time 91107) -> Assertion `patch_ok' failed to hold. The tracee issues a vsyscall, but we failed to moneypatch the caller. Recording will not succeed. Exiting. Tail of trace dump: { real_time:66044.530806 global_time:91087, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:78878, ticks:2408613 rax:0x7fb6266ae000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x210808 rdi:0x0 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fb64d2ffa80 gs_base:0x0 { map_file:"<ZERO>", addr:0x7fb6266ae000, length:0x211000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x211000 } } { real_time:66044.530863 global_time:91088, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:78878, ticks:2408695 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x20000 rdi:0x7fb64e2ff000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fb64d2ffa80 gs_base:0x0 } { real_time:66044.530915 global_time:91089, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:78878, ticks:2408695 rax:0x7fb64e2ff000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x20000 rdi:0x7fb64e2ff000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fb64d2ffa80 gs_base:0x0 { map_file:"<ZERO>", addr:0x7fb64e2ff000, length:0x20000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x20000 } } { real_time:66044.530967 global_time:91090, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:78878, ticks:2408730 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x7fb64d13f000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fb64d2ffa80 gs_base:0x0 } { real_time:66044.531022 global_time:91091, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:78878, ticks:2408730 rax:0x7fb64d13f000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x7fb64d13f000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fb64d2ffa80 gs_base:0x0 { map_file:"<ZERO>", addr:0x7fb64d13f000, length:0x1000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x1000 } } { real_time:66044.531074 global_time:91092, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:78878, ticks:2408765 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x7fb64ccc5000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fb64d2ffa80 gs_base:0x0 } { real_time:66044.531130 global_time:91093, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:78878, ticks:2408765 rax:0x7fb64ccc5000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x7fb64ccc5000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fb64d2ffa80 gs_base:0x0 { map_file:"<ZERO>", addr:0x7fb64ccc5000, length:0x1000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x1000 } } { real_time:66044.531181 global_time:91094, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:78878, ticks:2408800 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x7fb64a8ef000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fb64d2ffa80 gs_base:0x0 } { real_time:66044.531236 global_time:91095, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:78878, ticks:2408800 rax:0x7fb64a8ef000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x7fb64a8ef000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fb64d2ffa80 gs_base:0x0 { map_file:"<ZERO>", addr:0x7fb64a8ef000, length:0x1000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x1000 } } { real_time:66044.531287 global_time:91096, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:78878, ticks:2408835 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x7fb638a3f000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fb64d2ffa80 gs_base:0x0 } { real_time:66044.531342 global_time:91097, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:78878, ticks:2408835 rax:0x7fb638a3f000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x7fb638a3f000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fb64d2ffa80 gs_base:0x0 { map_file:"<ZERO>", addr:0x7fb638a3f000, length:0x1000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x1000 } } { real_time:66044.531394 global_time:91098, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:78878, ticks:2408882 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x100000 rdi:0x0 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fb64d2ffa80 gs_base:0x0 } { real_time:66044.531445 global_time:91099, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:78878, ticks:2408882 rax:0x7fb6265ae000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x100000 rdi:0x0 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fb64d2ffa80 gs_base:0x0 { map_file:"<ZERO>", addr:0x7fb6265ae000, length:0x100000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x100000 } } { real_time:66044.531518 global_time:91100, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:78878, ticks:2409503 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x10000 rdi:0x0 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x55de8eb5dca0 r13:0x12 r14:0x80c000000000 r15:0x80c000001fff rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fb64d2ffa80 gs_base:0x0 } { real_time:66044.531569 global_time:91101, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:78878, ticks:2409503 rax:0x7fb62659e000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x10000 rdi:0x0 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x55de8eb5dca0 r13:0x12 r14:0x80c000000000 r15:0x80c000001fff rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fb64d2ffa80 gs_base:0x0 { map_file:"<ZERO>", addr:0x7fb62659e000, length:0x10000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x10000 } } { real_time:66044.531627 global_time:91102, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:78878, ticks:2409567 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x10000 rdi:0x0 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x0 r13:0x12 r14:0x80c000000000 r15:0x80c000001fff rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fb64d2ffa80 gs_base:0x0 } { real_time:66044.531679 global_time:91103, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:78878, ticks:2409567 rax:0x7fb62658e000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x10000 rdi:0x0 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x0 r13:0x12 r14:0x80c000000000 r15:0x80c000001fff rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fb64d2ffa80 gs_base:0x0 { map_file:"<ZERO>", addr:0x7fb62658e000, length:0x10000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x10000 } } { real_time:66044.531756 global_time:91104, event:`INSTRUCTION_TRAP' tid:78878, ticks:2412555 rax:0xd rbx:0x756e6547 rcx:0x6c65746e rdx:0x49656e69 rsi:0x55de8e63f6c0 rdi:0xc00000c1e0 rbp:0x7ffe0426d088 rsp:0x7ffe0426d050 r8:0x7fb6266ae60e r9:0x203000 r10:0x8 r11:0x75 r12:0xf5 r13:0x0 r14:0x55de8e4299c8 r15:0x0 rip:0x55de8db35f5a eflags:0x10246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7fb64d2ffa80 gs_base:0x0 } { real_time:66044.531804 global_time:91105, event:`INSTRUCTION_TRAP' tid:78878, ticks:2412556 rax:0x306c3 rbx:0x100800 rcx:0x3ffafbbf rdx:0xbfebfbff rsi:0x55de8e63f6c0 rdi:0xc00000c1e0 rbp:0x7ffe0426d088 rsp:0x7ffe0426d050 r8:0x7fb6266ae60e r9:0x203000 r10:0x8 r11:0x75 r12:0xf5 r13:0x0 r14:0x55de8e4299c8 r15:0x0 rip:0x55de8db35f5a eflags:0x10206 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7fb64d2ffa80 gs_base:0x0 } { real_time:66044.531852 global_time:91106, event:`INSTRUCTION_TRAP' tid:78878, ticks:2412559 rax:0x0 rbx:0x27ab rcx:0x0 rdx:0x9c000600 rsi:0x55de8e63f6c0 rdi:0xc00000c1e0 rbp:0x7ffe0426d088 rsp:0x7ffe0426d050 r8:0x7fb6266ae60e r9:0x203000 r10:0x8 r11:0x75 r12:0xf5 r13:0x0 r14:0x55de8e4299c8 r15:0x0 rip:0x55de8db35f5a eflags:0x10206 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7fb64d2ffa80 gs_base:0x0 } === Start rr backtrace: /nix/store/6cglxyfwblnb1jpiqa2j008sm8j80q7z-rr-unstable-2020-10-10/bin/rr(_ZN2rr13dump_rr_stackEv+0x44)[0x69e873] /nix/store/6cglxyfwblnb1jpiqa2j008sm8j80q7z-rr-unstable-2020-10-10/bin/rr(_ZN2rr9GdbServer15emergency_debugEPNS_4TaskE+0x1a2)[0x4d0896] /nix/store/6cglxyfwblnb1jpiqa2j008sm8j80q7z-rr-unstable-2020-10-10/bin/rr[0x500834] /nix/store/6cglxyfwblnb1jpiqa2j008sm8j80q7z-rr-unstable-2020-10-10/bin/rr(_ZN2rr21EmergencyDebugOstreamD1Ev+0x62)[0x500a8e] /nix/store/6cglxyfwblnb1jpiqa2j008sm8j80q7z-rr-unstable-2020-10-10/bin/rr(_ZN2rr13RecordSession29handle_seccomp_traced_syscallEPNS_10RecordTaskEPNS0_9StepStateEPNS0_12RecordResultEPb+0x479)[0x5414b7] /nix/store/6cglxyfwblnb1jpiqa2j008sm8j80q7z-rr-unstable-2020-10-10/bin/rr(_ZN2rr13RecordSession19handle_ptrace_eventEPPNS_10RecordTaskEPNS0_9StepStateEPNS0_12RecordResultEPb+0x31a)[0x54262a] /nix/store/6cglxyfwblnb1jpiqa2j008sm8j80q7z-rr-unstable-2020-10-10/bin/rr(_ZN2rr13RecordSession11record_stepEv+0x31f)[0x54a97b] /nix/store/6cglxyfwblnb1jpiqa2j008sm8j80q7z-rr-unstable-2020-10-10/bin/rr[0x53c807] /nix/store/6cglxyfwblnb1jpiqa2j008sm8j80q7z-rr-unstable-2020-10-10/bin/rr(_ZN2rr13RecordCommand3runERSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x3dd)[0x53d44b] /nix/store/6cglxyfwblnb1jpiqa2j008sm8j80q7z-rr-unstable-2020-10-10/bin/rr(main+0x227)[0x6b9e1f] /nix/store/hp8wcylqr14hrrpqap4wdrwzq092wfln-glibc-2.32-37/lib/libc.so.6(__libc_start_main+0xed)[0x7fe5a0353ded] /nix/store/6cglxyfwblnb1jpiqa2j008sm8j80q7z-rr-unstable-2020-10-10/bin/rr(_start+0x2a)[0x43056a] === End rr backtrace Launch gdb with gdb '-l' '10000' '-ex' 'set sysroot /' '-ex' 'target extended-remote 127.0.0.1:13342' /usr/bin/snapAttach gdb to the tracee following the instructions at the bottom and get a backtrace?
When I run
gdb '-l' '10000' '-ex' 'set sysroot /' '-ex' 'target extended-remote 127.0.0.1:30174' /usr/bin/snapI get taken to
Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/ld-2.32.so... 0x000056040c49c20c in _start () from /lib64/ld-linux-x86-64.so.2 (gdb) bt #0 0x000056040c49c20c in _start () from /lib64/ld-linux-x86-64.so.2 #1 0x000056040cdcee20 in ?? () #2 0x000000c000002001 in ?? () #3 0x000000c00000c000 in ?? () #4 0x000056040cb95ea0 in ?? () #5 0x00007ffe52a173a8 in ?? () #6 0x000056040c47b039 in ?? () #7 0x0000000000010000 in ?? () #8 0x000056040cb95ea0 in ?? () #9 0x0000000000000004 in ?? () #10 0x000056040d4cd3ab in ?? () #11 0x0000000000000010 in ?? () #12 0x00007ffe52a173f0 in ?? () #13 0x000056040c466cac in ?? () #14 0x000056040d4cd240 in ?? () #15 0x0000000000000080 in ?? () #16 0x0000000000000080 in ?? () #17 0x0000000000000010 in ?? () #18 0x0000000000000000 in ?? ()Pressing
cjust results in termination of the program:(gdb) r The program being debugged has been started already. Start it from the beginning? (y or n) n Program not restarted. (gdb) c Continuing. Warning: Cannot insert breakpoint -2: Remote connection closed Command aborted.If you rerun that recording and attach the emergency debugger again, what's the value of
$rip, what's the return address on the stack at[$rsp], and what's the disassembly of the code at that address?Some output from gdb:
(gdb) p $rip $1 = (void (*)()) 0x55e0662bf20c <_start> (gdb) p $rsp $2 = (void *) 0x7ffe96ad7210 (gdb) x/10i $rsp 0x7ffe96ad7210: and %bl,(%rsi) 0x7ffe96ad7212: mov $0x55e066,%edi 0x7ffe96ad7217: add %al,(%rcx) 0x7ffe96ad7219: and %al,(%rax) 0x7ffe96ad721b: add %al,%al 0x7ffe96ad721d: add %al,(%rax) 0x7ffe96ad721f: add %al,(%rax) 0x7ffe96ad7221: rolb $0x0,(%rax) 0x7ffe96ad7224: rolb $0x0,(%rax) 0x7ffe96ad7227: add %ah,-0x1f996472(%rax) (gdb) x/10i $rip => 0x55e0662bf20c <_start>: mov (%rsp),%eax 0x55e0662bf20f <_start+3>: mov 0x8(%rsp),%edx 0x55e0662bf213 <_start+7>: mov %rbp,%rsp 0x55e0662bf216 <_dl_start_user+2>: movq $0x0,0x328(%rbx) 0x55e0662bf221 <_dl_start_user+13>: imul $0x3e8,%rdx,%rdx 0x55e0662bf228 <_dl_start_user+20>: imul $0x3b9aca00,%rax,%rax 0x55e0662bf22f <_dl_start_user+27>: add %rdx,%rax 0x55e0662bf232 <_dl_start_user+30>: mov %rax,0x18(%rsp) 0x55e0662bf237 <_dl_start_user+35>: mov 0x8(%rsp),%rbp 0x55e0662bf23c <_dl_start_user+40>: add $0x10,%rsp779040e adds some more debug logging, can you pull it and rerun your failing test?
It seems deeply wrong that the return address should be
_starthere...Here's the updated log:
[FATAL /build/source/src/RecordSession.cc:347:handle_seccomp_traced_syscall()] (task 30031 (rec:30031) at time 87012) -> Assertion `patch_ok' failed to hold. The tracee issues a vsyscall to 0xffffffffff600000 but we failed to moneypatch the caller (return address 0x5618f2a7920b, sp=0x7ffdf2fe34e8). Recording will not succeed. Exiting. Tail of trace dump: { real_time:6140.719522 global_time:86992, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:30031, ticks:629286 rax:0x7fa5d33e1000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x210808 rdi:0x0 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fa5fa033080 gs_base:0x0 { map_file:"<ZERO>", addr:0x7fa5d33e1000, length:0x211000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x211000 } } { real_time:6140.719591 global_time:86993, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:30031, ticks:629368 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x20000 rdi:0x7fa5fa246000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fa5fa033080 gs_base:0x0 } { real_time:6140.719656 global_time:86994, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:30031, ticks:629368 rax:0x7fa5fa246000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x20000 rdi:0x7fa5fa246000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fa5fa033080 gs_base:0x0 { map_file:"<ZERO>", addr:0x7fa5fa246000, length:0x20000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x20000 } } { real_time:6140.719719 global_time:86995, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:30031, ticks:629403 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x7fa5f9e72000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fa5fa033080 gs_base:0x0 } { real_time:6140.719787 global_time:86996, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:30031, ticks:629403 rax:0x7fa5f9e72000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x7fa5f9e72000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fa5fa033080 gs_base:0x0 { map_file:"<ZERO>", addr:0x7fa5f9e72000, length:0x1000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x1000 } } { real_time:6140.719851 global_time:86997, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:30031, ticks:629438 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x7fa5f99f8000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fa5fa033080 gs_base:0x0 } { real_time:6140.719919 global_time:86998, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:30031, ticks:629438 rax:0x7fa5f99f8000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x7fa5f99f8000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fa5fa033080 gs_base:0x0 { map_file:"<ZERO>", addr:0x7fa5f99f8000, length:0x1000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x1000 } } { real_time:6140.719983 global_time:86999, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:30031, ticks:629473 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x7fa5f7622000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fa5fa033080 gs_base:0x0 } { real_time:6140.720051 global_time:87000, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:30031, ticks:629473 rax:0x7fa5f7622000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x7fa5f7622000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fa5fa033080 gs_base:0x0 { map_file:"<ZERO>", addr:0x7fa5f7622000, length:0x1000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x1000 } } { real_time:6140.720115 global_time:87001, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:30031, ticks:629508 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x7fa5e5772000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fa5fa033080 gs_base:0x0 } { real_time:6140.720183 global_time:87002, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:30031, ticks:629508 rax:0x7fa5e5772000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x1000 rdi:0x7fa5e5772000 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x32 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fa5fa033080 gs_base:0x0 { map_file:"<ZERO>", addr:0x7fa5e5772000, length:0x1000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x1000 } } { real_time:6140.720247 global_time:87003, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:30031, ticks:629555 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x100000 rdi:0x0 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fa5fa033080 gs_base:0x0 } { real_time:6140.720310 global_time:87004, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:30031, ticks:629555 rax:0x7fa5d32e1000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x100000 rdi:0x0 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x40 r13:0x40 r14:0x1 r15:0x6e43a318 rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fa5fa033080 gs_base:0x0 { map_file:"<ZERO>", addr:0x7fa5d32e1000, length:0x100000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x100000 } } { real_time:6140.720399 global_time:87005, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:30031, ticks:630176 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x10000 rdi:0x0 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x5618f3a3bca0 r13:0x12 r14:0x80c000000000 r15:0x80c000001fff rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fa5fa033080 gs_base:0x0 } { real_time:6140.720462 global_time:87006, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:30031, ticks:630176 rax:0x7fa5d32d1000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x10000 rdi:0x0 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x5618f3a3bca0 r13:0x12 r14:0x80c000000000 r15:0x80c000001fff rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fa5fa033080 gs_base:0x0 { map_file:"<ZERO>", addr:0x7fa5d32d1000, length:0x10000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x10000 } } { real_time:6140.720533 global_time:87007, event:`SYSCALL: mmap' (state:ENTERING_SYSCALL) tid:30031, ticks:630240 rax:0xffffffffffffffda rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x10000 rdi:0x0 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x0 r13:0x12 r14:0x80c000000000 r15:0x80c000001fff rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fa5fa033080 gs_base:0x0 } { real_time:6140.720596 global_time:87008, event:`SYSCALL: mmap' (state:EXITING_SYSCALL) tid:30031, ticks:630240 rax:0x7fa5d32c1000 rbx:0x681fffa0 rcx:0xffffffffffffffff rdx:0x3 rsi:0x10000 rdi:0x0 rbp:0x681ffec0 rsp:0x681ffe70 r8:0xffffffff r9:0x0 r10:0x22 r11:0x246 r12:0x0 r13:0x12 r14:0x80c000000000 r15:0x80c000001fff rip:0x70000002 eflags:0x246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0x9 fs_base:0x7fa5fa033080 gs_base:0x0 { map_file:"<ZERO>", addr:0x7fa5d32c1000, length:0x10000, prot_flags:"rw-p", file_offset:0x0, device:0, inode:0, data_file:"", data_offset:0x0, file_size:0x10000 } } { real_time:6140.720686 global_time:87009, event:`INSTRUCTION_TRAP' tid:30031, ticks:633222 rax:0xd rbx:0x756e6547 rcx:0x6c65746e rdx:0x49656e69 rsi:0x5618f351d6c0 rdi:0xc00000c1e0 rbp:0x7ffdf2fe3548 rsp:0x7ffdf2fe3510 r8:0x7fa5d33e160e r9:0x203000 r10:0x8 r11:0x75 r12:0xf5 r13:0x0 r14:0x5618f33079c8 r15:0x0 rip:0x5618f2a13f5a eflags:0x10246 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7fa5fa033080 gs_base:0x0 } { real_time:6140.720745 global_time:87010, event:`INSTRUCTION_TRAP' tid:30031, ticks:633223 rax:0x306c3 rbx:0x100800 rcx:0x3ffafbbf rdx:0xbfebfbff rsi:0x5618f351d6c0 rdi:0xc00000c1e0 rbp:0x7ffdf2fe3548 rsp:0x7ffdf2fe3510 r8:0x7fa5d33e160e r9:0x203000 r10:0x8 r11:0x75 r12:0xf5 r13:0x0 r14:0x5618f33079c8 r15:0x0 rip:0x5618f2a13f5a eflags:0x10206 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7fa5fa033080 gs_base:0x0 } { real_time:6140.720803 global_time:87011, event:`INSTRUCTION_TRAP' tid:30031, ticks:633226 rax:0x0 rbx:0x27ab rcx:0x0 rdx:0x9c000600 rsi:0x5618f351d6c0 rdi:0xc00000c1e0 rbp:0x7ffdf2fe3548 rsp:0x7ffdf2fe3510 r8:0x7fa5d33e160e r9:0x203000 r10:0x8 r11:0x75 r12:0xf5 r13:0x0 r14:0x5618f33079c8 r15:0x0 rip:0x5618f2a13f5a eflags:0x10206 cs:0x33 ss:0x2b ds:0x0 es:0x0 fs:0x0 gs:0x0 orig_rax:0xffffffffffffffff fs_base:0x7fa5fa033080 gs_base:0x0 } === Start rr backtrace: /nix/store/r4riwfhnx9is1809kwwywgkjzhkpr1q3-rr-unstable-2020-10-10/bin/rr(_ZN2rr13dump_rr_stackEv+0x44)[0x69d39d] /nix/store/r4riwfhnx9is1809kwwywgkjzhkpr1q3-rr-unstable-2020-10-10/bin/rr(_ZN2rr9GdbServer15emergency_debugEPNS_4TaskE+0x1a2)[0x4d080a] /nix/store/r4riwfhnx9is1809kwwywgkjzhkpr1q3-rr-unstable-2020-10-10/bin/rr[0x50088c] /nix/store/r4riwfhnx9is1809kwwywgkjzhkpr1q3-rr-unstable-2020-10-10/bin/rr(_ZN2rr21EmergencyDebugOstreamD1Ev+0x62)[0x500ae6] /nix/store/r4riwfhnx9is1809kwwywgkjzhkpr1q3-rr-unstable-2020-10-10/bin/rr(_ZN2rr13RecordSession29handle_seccomp_traced_syscallEPNS_10RecordTaskEPNS0_9StepStateEPNS0_12RecordResultEPb+0x4d6)[0x5415c2] /nix/store/r4riwfhnx9is1809kwwywgkjzhkpr1q3-rr-unstable-2020-10-10/bin/rr(_ZN2rr13RecordSession19handle_ptrace_eventEPPNS_10RecordTaskEPNS0_9StepStateEPNS0_12RecordResultEPb+0x31a)[0x542734] /nix/store/r4riwfhnx9is1809kwwywgkjzhkpr1q3-rr-unstable-2020-10-10/bin/rr(_ZN2rr13RecordSession11record_stepEv+0x377)[0x54aa8b] /nix/store/r4riwfhnx9is1809kwwywgkjzhkpr1q3-rr-unstable-2020-10-10/bin/rr[0x53c88e] /nix/store/r4riwfhnx9is1809kwwywgkjzhkpr1q3-rr-unstable-2020-10-10/bin/rr(_ZN2rr13RecordCommand3runERSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x3dd)[0x53d4f9] /nix/store/r4riwfhnx9is1809kwwywgkjzhkpr1q3-rr-unstable-2020-10-10/bin/rr(main+0x227)[0x6b894f] /nix/store/hp8wcylqr14hrrpqap4wdrwzq092wfln-glibc-2.32-37/lib/libc.so.6(__libc_start_main+0xed)[0x7ff962546ded] /nix/store/r4riwfhnx9is1809kwwywgkjzhkpr1q3-rr-unstable-2020-10-10/bin/rr(_start+0x2a)[0x43056a] === End rr backtrace Launch gdb with gdb '-l' '10000' '-ex' 'set sysroot /' '-ex' 'target extended-remote 127.0.0.1:30031' /usr/bin/snapCan you disassemble the code at 0xffffffffff600000?
@khuey Re-running the trace I get
[FATAL /build/source/src/RecordSession.cc:347:handle_seccomp_traced_syscall()] (task 4805 (rec:4805) at time 94687) -> Assertion `patch_ok' failed to hold. The tracee issues a vsyscall to 0xffffffffff600000 but we failed to moneypatch the caller (return address 0x55dc68b6020b, sp=0x7fffd94b0cc8). Recording will not succeed. Exiting. # ...and then
(gdb) x/10i 0xffffffffff600000 0xffffffffff600000: Cannot access memory at address 0xffffffffff600000Hmm, ok. The vsyscall page should be there ...
You could try booting your kernel with the parameter
vsyscall=noneand see if this goes away.✔️ This worked. Thank you.
Testing now to see if Pernosco accepts the trace.
16 remaining items
@georgewsinger: Can you please drop the output of
uname -a?That PLT thunk stuff is probably glibc only and probably not happening here according to the parts of the disassembly we've seen so far.
Most likely all that is required here is to get the disassembly of the code leading up to the vsyscall as well as after it, and use that to create a patch template that matches this call site.
$ uname -a Linux UbuntuBox 5.8.0-50-generic #56-Ubuntu SMP Mon Apr 12 17:18:36 UTC 2021 x86_64 x86_64 x86_64 GNU/LinuxCan you keep going on the last one until you get back to 0x55b39d8a720b?
With
[FATAL /build/source/src/RecordSession.cc:347:handle_seccomp_traced_syscall()] (task 588943 (rec:588943) at time 79573) -> Assertion `patch_ok' failed to hold. The tracee issues a vsyscall to 0xffffffffff600000 but we failed to moneypatch the caller (return address 0x55ba7e56520b, sp=0x7ffdfad84678). Recording will not succeed. Exiting.I'm now getting
...which looks like mangled/unreadable output?
Can you do x/100i for that?
I ran it again and got a similar result, but it seems to be more readable assembly if I use
/iinstead of/r?
idk what /r does, idk why roc told you to use that :)
Out of curiosity, what version of libc does this system have?
2.32:
$ ldd --version ldd ldd (Ubuntu GLIBC 2.32-0ubuntu3) 2.32 Copyright (C) 2020 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Written by Roland McGrath and Ulrich Drepper.Have you updated to 8a15f2a ?
- Just to drop that here: I have 2.17 and the issue that showed the same hex address (which we've seen in the time function disassembly) was fixed by the recent snapshot.
Have you updated to 8a15f2a ?
@rocallahan Just tried updating to latest tip, and the issue has gone away. 👍
