Skip to content

HDDS-16933. Speed up TestKeyValueContainer, TestEventBufferOverflow, TestBlockInputStream and TestEventQueue - #11481

Draft
smengcl wants to merge 4 commits into
apache:masterfrom
smengcl:HDDS-16933
Draft

smengcl wants to merge 4 commits into
apache:masterfrom
smengcl:HDDS-16933

Conversation

@smengcl

@smengcl smengcl commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Generated-by: Claude Code (Opus 5.5)

What changes were proposed in this pull request?

Four unit test classes spend most of their time on waits or test data that no assertion depends on. Test only change, one commit per class.

  • TestKeyValueContainer#testAutoCompactionSmallSstFile creates 400 containers with 500 keys each and then sleeps 7s. Since HDDS-12233 container import no longer generates SST files, so the test only asserts that none exist. Create 10 containers per volume and drop the sleep.
  • TestEventBufferOverflow sleeps 2.1s between retry attempts. Advance a MockClock past the retry delay instead, as TestReconTaskControllerImpl already does.
  • TestBlockInputStream waits the default 1s ozone.client.read.retry.interval before each retry. Set the interval to 0, except in testRefreshPipelineFunction, which asserts the retry delay log.
  • TestEventQueue uses a handler that sleeps 2s for each of 11 events, processed by a single thread. Sleep 500ms instead. The test checks after 500ms that between 1 and 10 events are scheduled, and the 11th is scheduled only after about 5s.

Test class time in CI, four recent master runs compared to a run of these changes on my fork:

Test class master (s) with change (s)
TestKeyValueContainer 58 to 61 34.6
TestEventBufferOverflow 39 10.7
TestBlockInputStream 21 3.3
TestEventQueue 23 6.2

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-16933

How was this patch tested?

🤖 Generated with Claude Code

smengcl and others added 4 commits October 11, 2026 21:51
…File

Container import no longer generates SST files, so the test only asserts that none exist. Create 10 containers per volume instead of 200 and drop the 7s wait for a compaction that has no input.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Advance a MockClock past the retry delay instead of sleeping 2.1s between attempts, as TestReconTaskControllerImpl already does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Use a zero read retry interval instead of the default 1s, except in testRefreshPipelineFunction which asserts the retry delay log.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The handler slept 2s for each of 11 events on a single thread. 500ms still leaves events pending when the test checks progress after 500ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings October 12, 2026 04:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

One or more issues must be addressed before approval.

1 open finding
What changed in this PR

This test-only PR reduces execution time by replacing real waits with virtual time and shortening unnecessary test workloads.

Changes:

  • Uses MockClock for Recon retry tests.
  • Reduces event-handler sleep and container test data volume.
  • Sets block read retry intervals to zero where delay is not under test.
File Description
hadoop-ozone/​recon/​src/​test/​java/​org/​apache/​hadoop/​ozone/​recon/​tasks/​TestEventBufferOverflow.java Updated as part of this pull request.
hadoop-hdds/​framework/​src/​test/​java/​org/​apache/​hadoop/​hdds/​server/​events/​TestEventQueue.java Updated as part of this pull request.
hadoop-hdds/​container-service/​src/​test/​java/​org/​apache/​hadoop/​ozone/​container/​keyvalue/​TestKeyValueContainer.java Updated as part of this pull request.
hadoop-hdds/​client/​src/​test/​java/​org/​apache/​hadoop/​hdds/​scm/​storage/​TestBlockInputStream.java Updated as part of this pull request.

🧠 Review effort: Lite


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants