Prevent empty NSQ batches and bound HTTP waits - #3744
Draft
arthurianresolve wants to merge 4 commits into
Draft
Conversation
arthurianresolve
marked this pull request as ready for review
July 23, 2026 23:13
arthurianresolve
marked this pull request as draft
July 24, 2026 21:27
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Improve
NSQMetricsReporterreliability and make its HTTP timeout configurable through each supported setup path:NSQMetricsReporterFactory.Buildermetrics.reporter.timeoutfor Spring servicesdw.metrics.reporter.timeout.millisfor CDI servicesRoot cause
flush()sent a request even when its buffer was empty. The Apache HTTP client also had no connection-request, connection, or socket timeout, so a slow NSQ endpoint could delay later reporting cycles. Service configuration paths could not pass a timeout to the NSQ builder.Impact
NSQ no longer receives empty
/mpubrequests. HTTP waits use a 10-second default limit, and a timed-out request does not prevent a later batch from being attempted. Java, Spring, and CDI deployments can override the shared limit.Configuration
The default timeout is 10 seconds for the connection-pool wait, connection attempt, and response wait.
withTimeout(...)on the NSQ builder.metrics.reporter.timeoutwith Spring duration syntax, such as10s,500ms, or a bare number of seconds.dw.metrics.reporter.timeout.millisto a positive millisecond value. The default-1leaves the reporter's own default unchanged.When the property is omitted, existing behavior uses the reporter's 10-second default. Setting it for an unsupported reporter fails with a clear configuration error. The original five-argument CDI producer overload remains available for source and binary compatibility.
Release sequencing
The new builder capability is in
metrics-reporter:3.0.1-SNAPSHOT, and the root and starter reactors point to that version so this draft can build the producer and consumers together.Before this PR is ready to merge, publish immutable
metrics-reporter:3.0.1and replace both snapshot version pins with3.0.1. Clean reactors that do not include the opt-inmetrics-reportermodule must not depend on the snapshot.Integration
This PR overlaps #3743 in
NSQMetricsReporter. Whichever PR merges second should rebase and retain both #3743's UTF-8 byte sizing/writes and this PR's empty-batch and timeout changes, then rerun themetrics-reportertests.Test
Windows host: Java 11.0.31 and Maven 3.9.16.
Passed:
metrics-reportermodule: 5 tests, 0 failures, 0 errors, 0 skipped.spring-boot-starter-datawavemodule: 43 tests, 0 failures, 0 errors, 0 skipped.datawave-ws-commonmodule: 68 tests, 0 failures, 0 errors, 0 skipped.MetricsCdiConfigurationTestpassed.Coverage includes:
Fixes #3732
Fixes #3753