Repository navigation
Implement handling of blocking syscalls #7
Description
Activity
I put together a PoC that signals the ptrace-er when a ptrace-ee has been context-switched a configurable number of times (we'd want that to be "1" for our purposes here). Initial testing suggests that the signals are being delivered as one would expect. The signaling works basically the same way as for the hpc interrupts, except that in our case here the trace-ee has to program the interrupt, from within the syscall buffer lib. There are some corner cases to figure out but this seems like it might work.
After a few more experiments, I'm now pretty confident we can make this work. My little test program is here. To make this "actually" work, we need to process the de-sched interrupts in the parent process and disarm the counter there, which is pretty straightforward but needs a bunch of nasty POSIX gloop to pull off. Will prove that out next, since that part must be bulletproof.
@rocallahan I should have asked this before I got started, but (i) was your approach about the same as this? (iI) if not, do you happen to have your old test program around?
You've already gone way beyond me --- I just had a little test program and manually ran "perf" to observe the progress of the context-switch counter.
The ptrace and UNIXbeard hackery is mostly working here, but I'm getting a seemingly-spurious desched notification that I don't understand yet. Or, there's an actual desched happening and I don't know where it's coming from. I have a hackaround that's not so bad and works, but will see if I can tell what's causing it.
The extra SIGIO doesn't seem to be an extra desched, because the switch counter is the same in the first and second. I also tried monitoring desched on syscalls that block for increasingly longer durations, and the extra SIGIO is always sent immediately after desched, not at arbitrary times. So I'm pretty close to ascribing this to magic for the time being and moving on.
Here's about the only hypothesis I have
- child gets descheduled, bumps counter to i and schedules SIGIO
- SIGIO notification "schedules" child, but it doesn't run
- child is being ptraced, so we "deschedule" child to notify parent and bump counter to i+1
- counter signal generated, but SIGIO is already pending so this one is queued
- parent is notified and sees counter value i+1
- parent stops delivery of first signal and shuts off counter
- second SIGIO dequeued and delivered, notififying parent (counter is off, so no pseudo-desched possible here)
- parent notified and sees counter value i+1 again
- parent stops delivery of second SIGIO and we continue on
If this is what's happening, then the current hackaround should be a reliable way of dealing with this.
(I also confirmed that the $ip doesn't advance during the pseudo-step, when the program being traced isn't blocked on a syscall.)
I have a PoC for "rpc syscalls", where a tracee requests that the tracer make syscall(s) on its behalf, here. I think this is the last piece of technology we need.
Why is that needed?
We have to do a pretty tricky dance to set up the cs counter and syscall buffer region across record/replay. During recording, the counter fd has to be shared with rr, but in replay it should be emulated. This requires letting rr know the fd number that's to be shared. In both record and replay, the buffer region has to be actually opened and mmap'd by both the tracee and rr. rr also has to know it's the syscall buffer initialization code it's talking to. The cleanest way I could think of to implement this is with a fake syscall; use an unallocated syscall number (like -42) to trap into rr, then let rr decide what to do. That's what the PoC does.
I think we could make this work by keying off syscall param pattern matching and so forth, but I'd rather get away from that.
Sounds reasonable.
In other words, instead of assuming various things about the sequence of mmap/socket/recv/etc calls and patterns of parameters thereto, we could instead do something like
int counterfd = perf_event_open(...); char* shmfile = "/dev/shm/rr-tracee-[tid]"; struct msghdr msg; // set up msg syscall(MAGIC_SYSCALL_BUF_INIT, counterfd, shmfile, &msg);
then rr decides what do with the params based on record vs. replay.
(Note to self: need to sigprocmask during this setup or things will get very hard.)
- added a commit that references this issue
on Jun 29, 2013 16 remaining items
This patch ended up being less straightforward than I was hoping, so I'll try to put up a wiki page with a walkthrough in the near future.
Hmm, so will we have to do these two extra ioctls around every potentially-blocking syscall? That's very unfortunate.
These are taking ~600-800ns each, which, unfortunately, ends up being small potatoes compared to the overhead of a traced syscall. >1us per syscall doesn't make me happy but it's not significantly impacting execution time in the traces I'm looking at.
- added 4 commits that reference this issue
on Jun 23, 2021 - added a commit that references this issue
on Jul 5, 2021 - added a commit that references this issue
on Apr 7, 2023 - added 5 commits that reference this issue
on Apr 20, 2023 - added a commit that references this issue
on Jul 10, 2023
From @rocallahan
This is a good idea because it doesn't introduce the concept of a "blocking syscall", which is a bit hard to define. Instead it handles all syscalls the same way. More details forthcoming.