Skip to content

DEBUG: diagnose flaky memory-pressure RRCC test on Windows - #30755

Closed
fmeum wants to merge 3 commits into
bazelbuild:masterfrom
fmeum:debug-rrcc-memory-pressure-windows
Closed

DEBUG: diagnose flaky memory-pressure RRCC test on Windows#30755
fmeum wants to merge 3 commits into
bazelbuild:masterfrom
fmeum:debug-rrcc-memory-pressure-windows

Conversation

@fmeum

@fmeum fmeum commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator

Debug only, do not merge. Opened to collect Windows CI data on the flakiness of testMemoryPressureRestartDuringCachedFetch.

Description

Three temporary changes:

  • RepositoryFetchFunction logs DEBUG_RESTART to the server log whenever a fetch is restarted after a memory-pressure compute state drop, and HighWaterMarkLimiter logs every single DEBUG_DROP (the existing statements are rate limited to one per 10s). The test can only observe restarts through the progress bar today, which is transient and may never be rendered, so a missing message in stderr does not tell us whether the fetch was actually interrupted.
  • The test runs all 5 poison attempts instead of stopping at the first successful one and dumps, per attempt, the distinct fetch progress lines, all warnings/errors and the restart/drop lines from the server log.
  • .bazelci/presubmit.yml is trimmed down to a single Windows task that runs only this test, 6 times, so that one CI run yields 30 samples.

Motivation

The test is flaky on Windows on master:

AssertionError: the cached fetch of my_repo was never both interrupted by a memory-pressure compute state drop and served from the cache

Build API Changes

No

Checklist

  • I have added tests for the new use cases (if any).
  • I have updated the documentation (if applicable).

Release Notes

RELNOTES: None

fmeum added 3 commits August 17, 2026 15:53
Not for merge. Adds ground-truth logging for fetch restarts and state
drops, dumps per-attempt diagnostics from the test, and trims the CI
config down to repeated Windows runs of the flaky test only.
@fmeum

fmeum commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator Author

Diagnosis complete: the flake is fixed in #30757, and the crash these runs uncovered in #30756.

@fmeum fmeum closed this Aug 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant