-
Notifications
You must be signed in to change notification settings - Fork 6.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deflake ThreadStatus related unit tests #12858
Conversation
cf5b5c3
to
9800dd5
Compare
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
// TODO: fix places where default IOOption() is used, which can cause | ||
// io_activity to not match thread operation. | ||
assert(io_activity == Env::IOActivity::kUnknown || | ||
options.io_activity == Env::IOActivity::kUnknown || |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We kind of rely on NOT having this exception "options.io_activity == Env::IOActivity::kUnknown" to find incorrectly passed in IOOptions.... Is there anyway to fix the UT instead? By the way I like the refactory.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I could change the sync point used in unit test to make it work. But I think it makes sense to keep the thread_operation as Flush/Compaction instead of setting it to UNKNOWN. I assume that setting the thread_operation to UNKNOWN is only there to pass the checks in stress test. One alternative is to set the IOActivity in ~WritableFileWriter() based on thread_operation, what do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But I think it makes sense to keep the thread_operation as Flush/Compaction instead of setting it to UNKNOWN
Right unfortunately we can't pass parameter into destructor
I assume that setting the thread_operation to UNKNOWN is only there to pass the checks in stress test.
Right
One alternative is to set the IOActivity in ~WritableFileWriter() based on thread_operation, what do you think?
Yeah without refactoring the cleanup out of destructor you may do that under dbg mode.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated to set IOActivity in destructor.
95af88d
to
f9c4a34
Compare
@cbi42 has updated the pull request. You must reimport the pull request before landing. |
@cbi42 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Summary: Unit tests
DBTest.ThreadStatusFlush
andDBTestWithParam.ThreadStatusSingleCompaction
have been flaky and fail with error messageOne cause for this is that before flush/compaction finishes, we will go through
~WritableFileWriter()
, either for WAL or SST file, and temporarily set thread_operation to UNKNOWN. This UNKNOWN thread operation seem to be there for some stress test verification. This PR fixes these tests by setting the IOActivity in ~WritableFileWriter() for debug build.Test plan: monitor future test failure.