Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Fix flaky tests due to open server connection #1670

Merged
merged 6 commits into from
Jan 27, 2023

Conversation

dplewis
Copy link
Member

@dplewis dplewis commented Jan 27, 2023

New Pull Request Checklist

Issue Description

Sometimes the job status is returned as running when it should be failed. The status is returned too quickly before the job failed.

Parse Cloud run failing job
  Message:
    Unhandled promise rejection: AssertionError [ERR_ASSERTION]: 'running' == 'failed'

I've taken initial steps to find the fix for this. Once a test finishes there shouldn't be any open connections to the server. If there are then we log them in the CI and fix the as we go. Hopefully we can find which test is leaking.

- Error: Timeout - Async function did not complete within 20000ms (set by jasmine.DEFAULT_TIMEOUT_INTERVAL)
        - MongoNotConnectedError: Client must be connected before running operations

I removed the beforeAll from ParseDistTest as I only recommend using only one beforeAll in the test suite (one already exists in helper.js). I ensured the headless browser closes and doesn't leave any connections to the server.

Approach

Refactor Cloud Code Jobs to wait for the job status to change.

TODOs before merging

  • Add tests
  • Add changes to documentation (guides, repository pages, in-code descriptions)

@parse-github-assistant
Copy link

parse-github-assistant bot commented Jan 27, 2023

Thanks for opening this pull request!

  • ❌ Please link an issue that describes the reason for this pull request, otherwise your pull request will be closed. Make sure to write it as Closes: #123 in the PR description, so I can recognize it.

@codecov
Copy link

codecov bot commented Jan 27, 2023

Codecov Report

Base: 99.89% // Head: 99.89% // No change to project coverage 👍

Coverage data is based on head (93e2a7c) compared to base (8f8e978).
Patch has no changes to coverable lines.

Additional details and impacted files
@@           Coverage Diff           @@
##            alpha    #1670   +/-   ##
=======================================
  Coverage   99.89%   99.89%           
=======================================
  Files          61       61           
  Lines        5977     5977           
  Branches     1369     1369           
=======================================
  Hits         5971     5971           
  Misses          6        6           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@dplewis dplewis requested a review from a team January 27, 2023 01:25
Copy link
Member

@mtrezza mtrezza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glad this is fixed, can we merge?

@mtrezza mtrezza changed the title ci: Flaky Test ci: Fix flaky tests due to open server connection Jan 27, 2023
@dplewis
Copy link
Member Author

dplewis commented Jan 27, 2023

@mtrezza This can be merged

@dplewis
Copy link
Member Author

dplewis commented Jan 27, 2023

@mtrezza The issue is reconfigureServer for sure or related to it as I mentioned here

Screen Shot 2023-01-27 at 10 04 24 AM

@dplewis
Copy link
Member Author

dplewis commented Jan 27, 2023

@mtrezza I also noticed that the Node 14 build takes 17 minutes because of the mongodb-runner being installed globally. Should we move it to the devDependencies to increase the speed.

Screen Shot 2023-01-27 at 10 57 32 AM

@mtrezza
Copy link
Member

mtrezza commented Jan 27, 2023

I guess we can move it; do you know what this is only affecting Node 14?

@dplewis
Copy link
Member Author

dplewis commented Jan 27, 2023

I have no idea, the server tests work just fine for Node 14.

@mtrezza
Copy link
Member

mtrezza commented Jan 27, 2023

It could be this:

- name: Fix git protocol for Node 14
if: ${{ startsWith(matrix.NODE_VERSION, '14.') }}
run: sudo git config --system url."https://github".insteadOf "ssh://git@github"

But this has been added because otherwise some dependencies won't download at all.

@mtrezza
Copy link
Member

mtrezza commented Jan 27, 2023

Is this ready for merge?

@dplewis
Copy link
Member Author

dplewis commented Jan 27, 2023

@mtrezza this is ready to merge once the 17 minute CI passes. I'll play around with the CI for node 14 to see if I can find a better fix

@mtrezza
Copy link
Member

mtrezza commented Jan 27, 2023

The 17min CI passed, I'll go ahead and merge

@mtrezza mtrezza merged commit 94e193b into parse-community:alpha Jan 27, 2023
@dplewis dplewis deleted the flaky-test-cloud branch January 27, 2023 19:43
@parseplatformorg
Copy link
Contributor

🎉 This change has been released in version 4.0.0-alpha.7

@parseplatformorg parseplatformorg added the state:released-alpha Released as alpha version label Jan 30, 2023
@parseplatformorg
Copy link
Contributor

🎉 This change has been released in version 4.0.1-beta.1

@parseplatformorg parseplatformorg added the state:released-beta Released as beta version label Jan 31, 2023
@parseplatformorg
Copy link
Contributor

🎉 This change has been released in version 4.0.1

@parseplatformorg parseplatformorg added the state:released Released as stable version label Jan 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
state:released Released as stable version state:released-alpha Released as alpha version state:released-beta Released as beta version
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants