Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kvserver: fix flaky work queue tests #113259

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

srosenberg
Copy link
Member

In [1], new cluster settings have been added, e.g., kv.replica_gc_queue.enabled, kv.raft_log_queue.enabled, to selectively disable work queues per store, at runtime.

The new unit tests use a dummy BoolSetting which aliases slot 0 of the global cluster setting registry. Thus, its value corresponds to whichever cluster setting is registered first during init(). Before the change in [2], the first registered cluster setting is bulkio.backup.merge_file_buffer_size, one with a non-zero value in slot 0.
After the change, it is keyvisualizer.enabled, one with a zero value in slot 0. Subsequently, a number of the unit tests failed because baseQueue.mu.disabled is now true instead of previously, false.

The change in this PR fixes the bug by registering a fresh dummy BoolSetting. We also rename disabledConfig to match the polarity used in conjunction with bq.SetDisabled.

[1] #104170
[2] #113113

Epic: none

Release note: None

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@srosenberg srosenberg requested review from a team, herkolategan and renatolabs and removed request for a team October 28, 2023 14:59
@srosenberg srosenberg marked this pull request as ready for review October 28, 2023 14:59
@srosenberg srosenberg requested a review from a team October 28, 2023 14:59
@srosenberg srosenberg requested a review from a team as a code owner October 28, 2023 14:59
Copy link
Contributor

@erikgrinaker erikgrinaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice find, thanks for the fix and rename!

Copy link
Collaborator

@herkolategan herkolategan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @renatolabs and @srosenberg)


pkg/kv/kvserver/queue_test.go line 670 at r1 (raw file):

	// Now check that a queue which doesn't require the system config can
	// successfully add and process a replica.
	bqNoSysCfg := makeTestBaseQueue("test2", testQueue, tc.store, queueConfig{

super nit: is there a reason for the name change here, just out of curiosity?

Copy link
Collaborator

@herkolategan herkolategan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @renatolabs and @srosenberg)


pkg/kv/kvserver/queue_test.go line 670 at r1 (raw file):

Previously, herkolategan (Herko Lategan) wrote…

super nit: is there a reason for the name change here, just out of curiosity?

I'm guessing it is to avoid a possible conflict with bqNeedsSysCfg := makeTestBaseQueue("test", testQueue ... above?

In [1], new cluster settings have been added, e.g.,
`kv.replica_gc_queue.enabled`, `kv.raft_log_queue.enabled`,
to selectively disable work queues per store, at runtime.

The new unit tests use a dummy `BoolSetting` which aliases
slot 0 of the global cluster setting `registry`. Thus, its
value corresponds to whichever cluster setting is registered
first during `init()`. Before the change in [2], the first
registered cluster setting is `bulkio.backup.merge_file_buffer_size`,
one with a non-zero value in slot 0.
After the change, it is `keyvisualizer.enabled`, one with a
zero value in slot 0. Subsequently, a number of the unit tests
failed because `baseQueue.mu.disabled` is now true instead
of previously, false.

The change in this PR fixes the bug by registering a fresh dummy
`BoolSetting`. We also rename `disabledConfig` to match the polarity
used in conjunction with `bq.SetDisabled`.

[1] cockroachdb#104170
[2] cockroachdb#113113

Epic: none

Release note: None
@srosenberg srosenberg force-pushed the sr/kvserver_flaky_cluster_setting branch from c5020c4 to 20b6972 Compare November 16, 2023 19:03
@srosenberg srosenberg added backport-23.1.x Flags PRs that need to be backported to 23.1 backport-23.2.x Flags PRs that need to be backported to 23.2. labels Nov 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-23.1.x Flags PRs that need to be backported to 23.1 backport-23.2.x Flags PRs that need to be backported to 23.2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants