Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement blocking eventfd #3939

Open
wants to merge 31 commits into
base: master
Choose a base branch
from
Open

Implement blocking eventfd #3939

wants to merge 31 commits into from

Conversation

tiif
Copy link
Contributor

@tiif tiif commented Oct 4, 2024

This PR

  • Implemented blocking for both read and write of eventfd
  • Added test for eventfd blocking read and write
  • Removed eventfd blocking tests from fail-dep
  • Added a new BlockReason::Eventfd

cc #3665

@tiif
Copy link
Contributor Author

tiif commented Oct 4, 2024

@rustbot author

@rustbot rustbot added the S-waiting-on-author Status: Waiting for the PR author to address review comments label Oct 4, 2024
@tiif
Copy link
Contributor Author

tiif commented Oct 5, 2024

Hmm...why is rustfmt complaining? Let me try rebase it instead.

Comment on lines 87 to 88
let mut blocked_read_tid = self.blocked_read_tid.borrow_mut();
*blocked_read_tid = Some(ecx.active_thread());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let mut blocked_read_tid = self.blocked_read_tid.borrow_mut();
*blocked_read_tid = Some(ecx.active_thread());
*self.blocked_read_tid.borrow_mut() = Some(ecx.active_thread());

You can save yourself a lot of RefCell panic trouble by doing the entire op in one go and thus not keeping the lock for the rest of the function

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be a vec? Multiple threads could write and thus end up being blocked. Maybe add a test for that

Copy link
Contributor Author

@tiif tiif Oct 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea I missed that and that's indeed a possible case. Multithreaded case is a bit more complicated, I need more time to think about this, but this is the case I am considering:

If there are main thread + 2 additional threads:
let the counter be u64::max, so any write happened after will block.
2. thread1 try to write, and get blocked.
3. thread2 try to write, also get blocked.
4. Main thread read. Now, will thread1 and thread2 both get unblocked?

If both are unblocked, and thread1 is executed first. if thread1 write a u64::max again, thread2 should be blocked again. But the current implementation won't allow this to happen because thread2 is currently calling the unblock callback.

It also doesn't make sense to only unblock one thread, because the moment read happens, both thread1 and thread2 indeed can write.

I will check what actually happened in real system.

Copy link
Contributor Author

@tiif tiif Oct 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think eventfd will unblock every thread at once. So something like this will happen:

If there are main thread + 2 additional threads:

  1. let the counter be (u64::max - 1), so any write happened after will block.
  2. thread1 try to write, and get blocked.
  3. thread2 try to write, also get blocked.
  4. main thread read, so all thread gets unblocked
  5. thread1 unblocked and writes (u64::max - 1)
  6. thread 2 unblocked and try to continue writing (u64::max - 1), it gets block again

So essentially, we need might need to check the counter value and continue to block the thread again in the blocking callback functions. I am not sure how we can approach this without overcomplicated it because if we need to block a thread multiple times, we might run into something like:

  1. block_thread
  2. in the blocking callback, block_thread
  3. In the second block_thread callback, block_thread again
    ... and it continues

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can have a fn eventfd_write that blocks the thread if necessary, and in the "unblock" callback calls eventfd_write again. Recursion is a thing. :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I think this will work. I feel this has an unfortunate consequence of making it more complex, but I couldn't see another solution for this.

src/shims/unix/linux/eventfd.rs Outdated Show resolved Hide resolved
@bors
Copy link
Collaborator

bors commented Oct 8, 2024

☔ The latest upstream changes (presumably #3951) made this pull request unmergeable. Please resolve the merge conflicts.

@tiif
Copy link
Contributor Author

tiif commented Oct 14, 2024

EDIT: Resolved, there is //@error-in-other-file: deadlock

@oli-obk When running /miri test eventfd_block_twice --bless, the error below is shown. Is there any way to annotate the error when the diagnostics occured outside the test file?

error: there were 1 unmatched diagnostics that occurred outside the testfile and had no pattern
    Error: deadlock: the evaluated program deadlocked

error: there were 1 unmatched diagnostics
 --> tests/fail-dep/libc/eventfd_block_twice.rs:1:1
  |
1 | //@only-target: linux
  | ^ Error: deadlock: the evaluated program deadlocked
  |

src/shims/unix/linux/eventfd.rs Outdated Show resolved Hide resolved
Comment on lines +220 to +231
let mut waiter = Vec::new();
let mut blocked_write_tid = eventfd.blocked_write_tid.borrow_mut();
while let Some(tid) = blocked_write_tid.pop() {
waiter.push(tid);
}
drop(blocked_write_tid);

waiter.sort();
waiter.dedup();
for thread_id in waiter {
ecx.unblock_thread(thread_id, BlockReason::Eventfd)?;
}
Copy link
Contributor Author

@tiif tiif Oct 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't think of a testcase that can actually produce thread id duplication in blocked_write/read_id, but I will just keep the dedup here first.

@tiif tiif marked this pull request as ready for review October 18, 2024 05:09
@tiif
Copy link
Contributor Author

tiif commented Oct 18, 2024

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Waiting for a review to complete and removed S-waiting-on-author Status: Waiting for the PR author to address review comments labels Oct 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Waiting for a review to complete
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants