Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix intermittent test failures #1495

Closed
jung-kim opened this issue Oct 26, 2021 · 3 comments
Closed

Fix intermittent test failures #1495

jung-kim opened this issue Oct 26, 2021 · 3 comments

Comments

@jung-kim
Copy link
Collaborator

Test fails intermittently quite often and considering various combinations of tests we run, test usually never passes.

https://github.com/FredrikNoren/ungit/runs/4005712725?check_suite_focus=true

This maybe simple as doing more retries during tests

Screen Shot 2021-10-25 at 10 50 35 PM

@campersau
Copy link
Collaborator

I know macos clicktests are very flaky with GitHub Actions. On travis (which we need to migrate to .com) they often pass like here https://travis-ci.org/github/FredrikNoren/ungit/builds/772284376. That's why I have added a retry for clicks in 934d152:

async click(selector, clickCount) {
let elementHandle = await this.waitForElementVisible(selector);
try {
await elementHandle.click({ clickCount: clickCount });
} catch (err1) {
elementHandle = await this.waitForElementVisible(selector);
try {
await elementHandle.click({ clickCount: clickCount }); // try click a second time to reduce test flakiness
} catch (err2) {
winston.error(`Failed to click element: ${selector}`);
throw err2;
}
}
}

But since I don't have a mac these issues are very hard to debug for me.

@jung-kim
Copy link
Collaborator Author

I think number of tests has gotten bit out of control. I don't there is a much value in testing in so many node versions and OS versions. To me, node works on an OS, then it should be enough. This is because if a test fails on specific OS, then it's a library or node issue, not ungit issue.

If we were to have tests within different browsers, then that would be helpful. But not various tests for different OSs.

I think we should cut down on appveyor and travis and stick with github actions and make it stable

jung-kim added a commit that referenced this issue Dec 4, 2021
# Conflicts:
#       source/server.js
jung-kim added a commit that referenced this issue Dec 17, 2021
jung-kim added a commit that referenced this issue Feb 3, 2022
# Conflicts:
#       components/stash/stash.js
#       package-lock.json
#       package.json
jung-kim added a commit that referenced this issue Feb 26, 2022
@jung-kim
Copy link
Collaborator Author

Closing this issues now. #1505 has more details on current state of click testing.

While there are still some outstanding race conditions, tests has stabilized quite a lot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants