Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the e2e tests not land blocking #39129

Closed
wants to merge 1 commit into from

Conversation

cipolleschi
Copy link
Contributor

@cipolleschi cipolleschi commented Aug 23, 2023

Summary

Given that the e2e tests are quite flaky, we want to reduce the flakyness making them non land blocking.
The idea is to:

  1. always have them green in CircleCI
  2. store the test logs in CircleCI artifacts, so we can explore them if needed.
  3. use danger to read the artifacts from CircleCI and post a message to inform the user whether the Test ends up successfully or not.

Changelog:

[Internal] - Make e2e tests non land-blocking. Use Danger to report the status of the E2E jobs

Test Plan:

CircleCI stays green, logs are uploaded as artifacts.

@facebook-github-bot facebook-github-bot added CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. p: Facebook Partner: Facebook Partner labels Aug 23, 2023
@cipolleschi cipolleschi force-pushed the cipolleschi/make_e2e_test_not_blocking branch 2 times, most recently from a8926d3 to 1867b65 Compare August 23, 2023 14:42
@analysis-bot
Copy link

analysis-bot commented Aug 23, 2023

Platform Engine Arch Size (bytes) Diff
android hermes arm64-v8a 8,963,889 -1,646
android hermes armeabi-v7a n/a --
android hermes x86 n/a --
android hermes x86_64 n/a --
android jsc arm64-v8a 9,553,297 -1,126
android jsc armeabi-v7a n/a --
android jsc x86 n/a --
android jsc x86_64 n/a --

Base commit: 2692f20
Branch: main

@cipolleschi cipolleschi force-pushed the cipolleschi/make_e2e_test_not_blocking branch 2 times, most recently from 1b01ac5 to ab1572c Compare August 24, 2023 10:05
@cipolleschi cipolleschi marked this pull request as ready for review August 24, 2023 10:10
@facebook-github-bot
Copy link
Contributor

@cipolleschi has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

1 similar comment
@facebook-github-bot
Copy link
Contributor

@cipolleschi has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@cipolleschi cipolleschi force-pushed the cipolleschi/make_e2e_test_not_blocking branch from ab1572c to 683aee8 Compare August 24, 2023 13:25
@github-actions
Copy link

This pull request was successfully merged by @cipolleschi in a058586.

When will my fix make it into a release? | Upcoming Releases

@github-actions github-actions bot added the Merged This PR has been merged. label Aug 25, 2023
@cortinico cortinico deleted the cipolleschi/make_e2e_test_not_blocking branch June 10, 2024 13:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Merged This PR has been merged. p: Facebook Partner: Facebook Partner
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants