Skip to content

ci: Fix flaky tests due to open server connection - #1670

Merged
mtrezza merged 6 commits into
parse-community:alphafrom
dplewis:flaky-test-cloud
Jan 27, 2023
Merged

mtrezza merged 6 commits into
parse-community:alphafrom
dplewis:flaky-test-cloud

Conversation

@dplewis

@dplewis dplewis commented Jan 27, 2023

Copy link
Copy Markdown
Member

New Pull Request Checklist

Issue Description

Sometimes the job status is returned as running when it should be failed. The status is returned too quickly before the job failed.

Parse Cloud run failing job
  Message:
    Unhandled promise rejection: AssertionError [ERR_ASSERTION]: 'running' == 'failed'

I've taken initial steps to find the fix for this. Once a test finishes there shouldn't be any open connections to the server. If there are then we log them in the CI and fix the as we go. Hopefully we can find which test is leaking.

- Error: Timeout - Async function did not complete within 20000ms (set by jasmine.DEFAULT_TIMEOUT_INTERVAL)
        - MongoNotConnectedError: Client must be connected before running operations

I removed the beforeAll from ParseDistTest as I only recommend using only one beforeAll in the test suite (one already exists in helper.js). I ensured the headless browser closes and doesn't leave any connections to the server.

Approach

Refactor Cloud Code Jobs to wait for the job status to change.

TODOs before merging

  • Add tests
  • Add changes to documentation (guides, repository pages, in-code descriptions)

@parse-github-assistant

parse-github-assistant Bot commented Jan 27, 2023

Copy link
Copy Markdown

Thanks for opening this pull request!

  • ❌ Please link an issue that describes the reason for this pull request, otherwise your pull request will be closed. Make sure to write it as Closes: #123 in the PR description, so I can recognize it.

@codecov

codecov Bot commented Jan 27, 2023

Copy link
Copy Markdown

Codecov Report

Base: 99.89% // Head: 99.89% // No change to project coverage 👍

Coverage data is based on head (93e2a7c) compared to base (8f8e978).
Patch has no changes to coverable lines.

Additional details and impacted files
@@           Coverage Diff           @@
##            alpha    #1670   +/-   ##
=======================================
  Coverage   99.89%   99.89%           
=======================================
  Files          61       61           
  Lines        5977     5977           
  Branches     1369     1369           
=======================================
  Hits         5971     5971           
  Misses          6        6           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@dplewis
dplewis requested a review from a team January 27, 2023 01:25

@mtrezza mtrezza left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glad this is fixed, can we merge?

@mtrezza mtrezza changed the title ci: Flaky Test ci: Fix flaky tests due to open server connection Jan 27, 2023
@dplewis

dplewis commented Jan 27, 2023

Copy link
Copy Markdown
Member Author

@mtrezza This can be merged

@dplewis

dplewis commented Jan 27, 2023

Copy link
Copy Markdown
Member Author

@mtrezza The issue is reconfigureServer for sure or related to it as I mentioned here

Screen Shot 2023-01-27 at 10 04 24 AM

@dplewis

dplewis commented Jan 27, 2023

Copy link
Copy Markdown
Member Author

@mtrezza I also noticed that the Node 14 build takes 17 minutes because of the mongodb-runner being installed globally. Should we move it to the devDependencies to increase the speed.

Screen Shot 2023-01-27 at 10 57 32 AM

@mtrezza

mtrezza commented Jan 27, 2023

Copy link
Copy Markdown
Member

I guess we can move it; do you know what this is only affecting Node 14?

@dplewis

dplewis commented Jan 27, 2023

Copy link
Copy Markdown
Member Author

I have no idea, the server tests work just fine for Node 14.

@mtrezza

mtrezza commented Jan 27, 2023

Copy link
Copy Markdown
Member

It could be this:

- name: Fix git protocol for Node 14
if: ${{ startsWith(matrix.NODE_VERSION, '14.') }}
run: sudo git config --system url."https://github".insteadOf "ssh://git@github"

But this has been added because otherwise some dependencies won't download at all.

@mtrezza

mtrezza commented Jan 27, 2023

Copy link
Copy Markdown
Member

Is this ready for merge?

@dplewis

dplewis commented Jan 27, 2023

Copy link
Copy Markdown
Member Author

@mtrezza this is ready to merge once the 17 minute CI passes. I'll play around with the CI for node 14 to see if I can find a better fix

@mtrezza

mtrezza commented Jan 27, 2023

Copy link
Copy Markdown
Member

The 17min CI passed, I'll go ahead and merge

@mtrezza
mtrezza merged commit 94e193b into parse-community:alpha Jan 27, 2023
@dplewis
dplewis deleted the flaky-test-cloud branch January 27, 2023 19:43
@parseplatformorg

Copy link
Copy Markdown
Contributor

🎉 This change has been released in version 4.0.0-alpha.7

@parseplatformorg parseplatformorg added the state:released-alpha Released as alpha version label Jan 30, 2023
@parseplatformorg

Copy link
Copy Markdown
Contributor

🎉 This change has been released in version 4.0.1-beta.1

@parseplatformorg parseplatformorg added the state:released-beta Released as beta version label Jan 31, 2023
@parseplatformorg

Copy link
Copy Markdown
Contributor

🎉 This change has been released in version 4.0.1

@parseplatformorg parseplatformorg added the state:released Released as stable version label Jan 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state:released Released as stable version state:released-alpha Released as alpha version state:released-beta Released as beta version

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants