Skip to content

Fix flaky signing_in ui test #66664

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: staging
Choose a base branch
from

Conversation

alex-m-brown
Copy link
Contributor

@alex-m-brown alex-m-brown commented Jun 23, 2025

The iPhone test Scenario: Student sign in from code.org flaked on Tue 3/18/25 with unexpected token at 'dbf54315b9544c8e19aa2305' (JSON::ParserError). It seems like a rather common error with iPhone tests that are often caught on retries (example).

After running this test locally over a dozen times, I have been unable to repro. In order to reduce the chances of this test flaking in the future, I added a new step that retries clicking the sign-in button.

Links

Testing story

Verified the changes don't affect a successful test run. Unable to repro the initial issue that caused the flake.

Follow-up work

I will watch the DTT for a while to see how this test does and if it flakes again.

@alex-m-brown alex-m-brown marked this pull request as ready for review June 24, 2025 13:21
@alex-m-brown alex-m-brown changed the title flakiness fix attempt Fix flaky signing_in ui test Jun 24, 2025
@alex-m-brown alex-m-brown requested a review from a team June 24, 2025 16:13
Copy link
Contributor

@nicklathe nicklathe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't written many UI tests, but this seems like a reasonable fix to me 👍

@@ -13,7 +12,7 @@ Scenario: Student sign in from code.org
Then I click "#header_user_signin"
And I wait to see "#signin"
And I fill in username and password for "Bob"
And I click "#signin-button" to load a new page
And I click the sign-in button to load a new page
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe a single retry should be sufficient in this case:

Suggested change
And I click the sign-in button to load a new page
And I press the first "#signin-button" element to load a new page

When /^I press the first "([^"]*)" element(?: to load a new (page|tab))?$/ do |selector, load|
wait_short_until do
@element = @browser.find_element(:css, selector)
end
page_load(load) do
@element.click
rescue
# Single retry to compensate for element changing between find and click
@element = @browser.find_element(:css, selector)
@element.click
end
end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants