Flaky Test - prometheusexporter.TestEndToEndSummarySupport

[djaglowski]

Seen here.

=== RUN   TestEndToEndSummarySupport
    end_to_end_test.go:182: Left-over unmatched Prometheus scrape content: "test_jvm_gc_collection_seconds_count{gc=\"G1 Old Generation\"} 9.223372036854776e+18 1636565705608test_jvm_info{vendor=\"Oracle Corporation\",version=\"9.0.4+11\"} NaN 1636565705608test_jvm_memory_pool_bytes_used{pool=\"CodeHeap 'non-nmethods'\"} NaN 1636565705608test_jvm_memory_pool_bytes_used{pool=\"CodeHeap 'non-profiled nmethods'\"} NaN 1636565705608test_jvm_memory_pool_bytes_used{pool=\"CodeHeap 'profiled nmethods'\"} NaN 1636565705608test_jvm_memory_pool_bytes_used{pool=\"Compressed Class Space\"} NaN 1636565705608test_jvm_memory_pool_bytes_used{pool=\"G1 Eden Space\"} NaN 1636565705608test_jvm_memory_pool_bytes_used{pool=\"G1 Old Gen\"} NaN 1636565705608test_jvm_memory_pool_bytes_used{pool=\"G1 Survivor Space\"} NaN 1636565705608test_jvm_memory_pool_bytes_used{pool=\"Metaspace\"} NaN 1636565705608test_scrape_samples_post_metric_relabeling 0 1636565705617test_scrape_samples_scraped 0 1636565705617test_scrape_series_added 0 1636565705617test_up 0 1636565705617"
--- FAIL: TestEndToEndSummarySupport (5.08s)

No code owner is currently assigned for this component.

[PaurushGarg]

@alolita please assign this issue to me. I would like to work on this one.

[jpkrohling]

Seen here as well: https://github.com/open-telemetry/opentelemetry-collector-contrib/runs/4457605222?check_suite_focus=true

[jpkrohling]

@PaurushGarg, do you have any progress to share?

[PaurushGarg]

@PaurushGarg, do you have any progress to share?

@jpkrohling I have raised a PR for this on open-o11y for approval by @Aneurysm9 before I raise it here. Link for OR: open-o11y#1026
As per my understanding, this is happening because intention of this test is to have HttpTest server respond (dropWizard) to atleast 8 requests -- but it only responds to first 8 requests because of the way select statement for waitForScrapes is configured. And if the last GET request by the Prometheus Receiver doesn't receive the response (dropWizard) from the HTTP test server, receiver would generate staleNaN (for first time no response is encountered). and that's why this test is flaky..

[dashpole]

This is fixed

[dmitryax]

It seems to be failing again with another message: #8365