chore: Fix flaky FlagSynchronizerSpec tests (SDK-2042) - #483
Merged
Conversation
Stop polling inside the onSyncComplete callback to prevent the timer from firing a second tick before the test can set isOnline = false. This is the same pattern applied in pollingTimerFiresSpec (PR #481). Fixes the race condition where getFeatureFlagsCallCount == 2 instead of 1 on slow CI runners. SDK-2042 Co-Authored-By: Aaron Zeisler <azeisler@launchdarkly.com>
The first waitUntil block used Nimble's default 1-second timeout, but the callback delivery chain (main RunLoop → syncQueue → getFeatureFlags → reportSyncComplete → DispatchQueue.main.async → onSyncComplete) can exceed 1 second on loaded CI runners. Increase to 5 seconds, consistent with other async tests in the codebase. SDK-2042 Co-Authored-By: Aaron Zeisler <azeisler@launchdarkly.com>
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
aaron-zeisler
marked this pull request as ready for review
March 17, 2026 17:33
tanderson-ld
approved these changes
Mar 18, 2026
aaron-zeisler
approved these changes
Mar 18, 2026
3 tasks
kinyoklion
added a commit
that referenced
this pull request
Mar 23, 2026
**Requirements** - [ ] I have added test coverage for new or changed functionality - [x] I have followed the repository's [pull request submission guidelines](../blob/v11/CONTRIBUTING.md#submitting-pull-requests) - [ ] I have validated my changes against all supported platform versions **Related issues** - Follows up on #483 and #480 which addressed other flaky tests in this file - Fixes the `build-ios (15.4.0, macos-14)` CI failure: `polling_timer_fires__one_second_interval__stops_polling, failed - expected to equal <2>, got <6>` - Fixes the `build-ios (16.4.0, macos-15)` CI failure: `streaming_events__event_reported_while_polling__reports_an_event_error, failed - waitUntil(..) expects its completion closure to be only called once` **Describe the solution you've provided** Adds a `didSignal` guard to prevent `onSyncComplete` callbacks from re-entering and double-signaling the semaphore. For tests that verify polling stopped, captures the call count after stopping and uses `Thread.sleep` to confirm no further requests occur. For tests that only need to confirm polling started, relaxes exact-count assertions to `>= 1`. Five tests are fixed: 1. **`changeIsOnlineSpec` ("stops polling")** — Added `didSignal` guard. The `isOnline = false` + `semaphore.signal()` inside the callback was already present in v11. Changed `== 1` to `>= 1`, then captures `countAfterStop`, sleeps 1.5s, and asserts count unchanged — proving polling actually stopped. 2. **`changeIsOnlineSpec` ("starts polling")** — Added `didSignal` guard to prevent double-signaling. No other changes to callback body; original test structure preserved (`isOnline` stays `true`, cleanup at the end). Changed `== 1` to `>= 1`. 3. **`changeIsOnlineSpec` ("does not stop polling")** — Added `didSignal` guard. No `isOnline = false` in callback; original test structure preserved. Changed `== 1` to `>= 1`. 4. **`streamingProcessingSpec` ("event reported while polling")** — Replaced the first `waitUntil` block with the semaphore + `didSignal` guard pattern. The original `waitUntil { done in ... { _ in done() } ... isOnline = true }` failed when polling timer ticks called `done()` more than once. 5. **`pollingTimerFiresSpec` ("stops polling")** — After stopping polling inside the callback at `requestCount == 2`, the test now captures the call count, waits 1.5 seconds, and asserts the count hasn't changed — directly proving polling actually stopped. Uses `>= 2` instead of `== 2` to tolerate in-flight callbacks. All changes are test-code-only; no application code is modified. **Describe alternatives you've considered** An earlier revision stopped polling (`isOnline = false`) inside the callbacks for tests #2 and #3 as well, but this introduced a timing dependency: the count staying at exactly 1 relied on the main queue processing the callback before the next 1-second timer tick. The current approach avoids that dependency entirely — the guard prevents re-signaling and `>= 1` tolerates any number of extra ticks. **Additional context** > **For reviewers — human review checklist:** > - **`>= 1` in "does not stop polling"**: This is weaker than the original `== 1`. The original assertion caught a restart regression (second `isOnline = true` triggering another immediate flag request). With `>= 1`, both normal polling and a restart regression satisfy the assertion. The `startPolling()` guard (`flagRequestTimer == nil`) in production code is what prevents restarts; this test no longer independently verifies that. If this is a concern, an alternative would be exposing timer state for direct inspection. > - **`didSignal` is not explicitly synchronized**, matching the existing `didCallDone` pattern in `LDTimerSpec`. The guard only needs to prevent the callback *body* from running a second time — it does not need to be atomic to accomplish this. > - **`Thread.sleep(forTimeInterval: 1.5)`** is used in both "stops polling" tests. This is generous relative to the 1-second polling interval but could theoretically be tight on very slow CI runners. Let me know if you'd prefer a longer wait or a different verification approach. > - **`pollingTimerFiresSpec` and `changeIsOnlineSpec` "stops polling" both use `>= N` then verify count stability**: There is a small window where an in-flight `getFeatureFlags` call could land between `isOnline = false` and the `countAfterStop` read. The `>= N` tolerates this, and the 1.5-second sleep assertion proves no *further* polling occurs regardless of the exact count at stop time. Link to Devin session: https://app.devin.ai/sessions/120fb9abea7743249e11b5bcbbcd1d8a Requested by: @kinyoklion --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
BEGIN_COMMIT_OVERRIDE
chore: Fix flaky FlagSynchronizerSpec tests
END_COMMIT_OVERRIDE
Requirements
Related issues
Describe the solution you've provided
Fixes two flaky tests in
FlagSynchronizerSpec.swift, each in a separate commit:change_isOnline__online_to_offline__stops_polling— MovedisOnline = falseinside theonSyncCompletecallback, before signaling the semaphore. This prevents a second timer tick from firing between the callback and the assertion, which causedgetFeatureFlagsCallCountto be 2 instead of 1 on slow CI runners. This is the same pattern already applied topollingTimerFiresSpecin PR chore: Parallelize builds to cut down CI wall clock time #481.streaming_events__event_reported_while_polling__reports_an_event_error— Added an explicit 5-second timeout to the firstwaitUntilblock. The default 1-second timeout was insufficient for the multi-hop async callback chain (main RunLoop → syncQueue → getFeatureFlags → reportSyncComplete → DispatchQueue.main.async) on loaded CI runners. The 5-second timeout is consistent with other async tests in the codebase (e.g.,DarklyServiceSpec,EventReporterSpec).Both changes are test-code-only; no application code is modified.
Describe alternatives you've considered
For fix 2, an alternative was converting to the semaphore + RunLoop pump pattern used elsewhere in
FlagSynchronizerSpec. However,waitUntil(timeout:)is the dominant async pattern in the test suite (16 call sites across 4 files), so an explicit timeout is more consistent and a smaller change.Additional context
Link to Devin session: https://app.devin.ai/sessions/1db84bb989a24b319cbc345be77efaac
Note
Low Risk
Test-only changes that adjust async coordination and timeouts; low risk aside from potentially masking real timing regressions if timeouts are too generous.
Overview
Fixes flakiness in
FlagSynchronizerSpecpolling-related tests by stopping polling inside theonSyncCompletecallback (before unblocking assertions) to avoid extra timer ticks racing the expectations.Also increases the first
waitUntiltimeout to 5 seconds for the “event reported while polling” case so the multi-hop async setup reliably completes on slower CI runners.Written by Cursor Bugbot for commit 2be732e. This will update automatically on new commits. Configure here.