Skip to content

fix: Flasky-tests #20413#20414

Draft
aadityarshah wants to merge 1 commit intoankidroid:mainfrom
aadityarshah:fix/flaky-tests
Draft

fix: Flasky-tests #20413#20414
aadityarshah wants to merge 1 commit intoankidroid:mainfrom
aadityarshah:fix/flaky-tests

Conversation

@aadityarshah
Copy link
Contributor

Purpose / Description

While working on my PR #20412, I faced failure in JUnitTests (macos run 1) and on reviewing it, I observed that no changes I had made during any of my commits affected the files in which errors were faced and on further reviewing, I realized that these were triggered due to the specific development environment (Linux/Windows dual-boot and non-UTC TimeZone. Hence, I raised the issue #20413. This PR aims to resolve the issue.

Fixes

Fixes #20413

Approach

This change addresses the following three categories of instabilities (all of those reported in the issue #20413):

  • ResourceID #0x0: Added a filter in the file PrefsSearchBarTest.kt in the directory AnkiDroid/src/test/java/com/ichi2/anki/preferences to ignore non-XML indexed items (IDs of 0), which were causing Resources$NotFoundException.
  • TimeZone Determinism: Modified MockTime.timestamp in the MockTime.kt inside directory common/src/main/java/com/ichi2/anki/common/time to use a forced GMT calendar and updated AlarmManagerServiceTest.kt inside the directory AnkiDroid/src/test/java/com/ichi2/anki/services/AlarmManagerServiceTest.kt to set a UTC default during execution which prevents assertions from failing in a non-UTC TimeZone.
  • Race Conditions: Replaced advanceRobolectricLooper() with ShadowLooper.runUiThreadTasksIncludingDelayedTasks() in ReviewerTest.kt wherever required and added advanceRobolectricLooper() where required in DeckPickerTest.kt, both of the files present in the directory AnkiDroid/src/test/java/com/ichi2/anki. This ensures the full asynchronous chain (Collection -> Scheduler -> WebView) completes before assertions run.

How Has This Been Tested?

  • Used OpenJDK version 24 to run ./gradlew testPlayDebugUnitTest which ran successfully on implementing the following changes allowing me to conclude that the changes were made successfully.

Learning (optional, can help others)

  • AI Disclosure: I used AI as a debugging consultant to help interpret the stack traces and understand Robolectric looper function. It helped me fix the issues in Race Conditions. I implemented and reviewed the changes manually ensuring there are no unnecessary changes in the code.
  • Insights: I learned about how virtual time works with Robolectric, as well as the need for proper looper flushes, especially with WebView-based activities like the Reviewer. I also learned more about how JNI binaries are handled by the build system, depending on the host OS architecture.

Checklist

Please, go through these checks before submitting the PR.

  • You have a descriptive commit message with a short title (first line, max 50 chars).
  • You have commented your code, particularly in hard-to-understand areas
  • You have performed a self-review of your own code
  • UI changes: include screenshots of all affected screens (in particular showing any new or changed strings)
  • UI Changes: You have tested your change using the Google Accessibility Scanner

This commit fixes the flaky-tests issue faced in the dual boot environment of Windows 11 and Ubuntu Linux caused due to TimeZone conditions and race conditions in ReviewerTest.kt and AlarmManagerServiceTest.kt
@Before
override fun setUp() {
super.setUp()
TimeZone.setDefault(TimeZone.getTimeZone("UTC"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't this fail at the same time in UTC?

Comment on lines +100 to +105
assertDoesNotThrow {
ChangeManager.notifySubscribers(
opChanges { studyQueues = true },
null,
)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be a no-op


(filesResIds + prefItemsResIds)
.filterIsInstance<Int>()
.filter { it != 0 }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is there sometimes a resource of 0?

It's best to figure this out, rather than ignore something in the test which we don't understand.

Is this 0 a missing resource, or something which was incorrectly loaded?

I think the best option would be to add a log of the count + names of all selected resource IDs, then we know if it's extra or missing.

val didDynamicA = addDynamicDeck("Deck Dynamic 1")

val noteEditor = selectContextMenuOptionForActivity(DeckPickerContextMenuOption.ADD_CARD, didA)
val noteEditor =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please revert all these spacing changes, they make it difficult to understand the change

equalFirstField(cards[0], reviewer.currentCard!!)
reviewer.answerCard(Rating.AGAIN)
advanceRobolectricLooper()
runUiThreadTasksIncludingDelayedTasks()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These appear to be equivalent

        fun advanceRobolectricLooper() {
            Shadows.shadowOf(Looper.getMainLooper()).runToEndOfTasks()
        }
  public static void runUiThreadTasksIncludingDelayedTasks() {
    getShadowMainLooper().runToEndOfTasks();
  }

@david-allison david-allison added Needs Author Reply Waiting for a reply from the original author and removed Needs Review labels Mar 8, 2026
@aadityarshah
Copy link
Contributor Author

@david-allison Thank you for your review, I am moving this PR to draft while I work through your feedback and address the necessary changes. I will update you once I am ready and will make the PR open once updates are done.

@aadityarshah aadityarshah marked this pull request as draft March 8, 2026 10:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Needs Author Reply Waiting for a reply from the original author

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky-tests: testPlayDebugUnitTest failure due to TimeZone assumptions and race conditions in ReviewerTest/AlarmManagerServiceTest

2 participants