Replace Jest runner with Wdio runner E2E

[Othinn]

Summary:

Due to the high flakiness of e2e tests, we suggested changing the Jest runner to Wdio runner, which should help us reduce the flakiness and increase the usefulness of the tests through better logs.

Changelog:

• Changed test runner from Jest to Wdio with Mocha assertion library. Because of using Mocha, I changed the Jest test block to Mochas it and changed the arrow functions to regular functions (in Mocha, the arrow function is discouraged from using https://mochajs.org/#arrow-functions)
• All tests are moved to component folders(naming convention of folders and tests to be decided)
• Moved steps to separate it for better visibility on results
• As a temporal solution, I've created separate conf files for ios and Android
• Added spec reporter. After we decide about result storing, we can change to, eg. Allure
• Appium is added to services, which means that we don't start the appium server by a terminal, and Appium logs are displayed in a terminal while tests are running
• Updated CircleCI jobs

Test Plan:

[facebook-github-bot]

Hi @Othinn!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

[analysis-bot]

Platform	Engine	Arch	Size (bytes)	Diff
android	hermes	arm64-v8a	16,777,794	-173,275
android	hermes	armeabi-v7a	n/a	--
android	hermes	x86	n/a	--
android	hermes	x86_64	n/a	--
android	jsc	arm64-v8a	20,165,571	-169,275
android	jsc	armeabi-v7a	n/a	--
android	jsc	x86	n/a	--
android	jsc	x86_64	n/a	--

Base commit: 96ed119
Branch: main

[samuelfreiberg]

I would suggest changing this scroll logic completely. Taking the window size and performing mathematical operations on it to determine touches is prone to error. Here's a better way:

    await browser.waitUntil(
      async () => {
        await driver.execute('mobile: scroll', { direction: 'down' });
        return await $(locator).isDisplayed();
      },
    );

[Othinn]

I've tried to change scroll logic with your or similar solutions, but after testing, the current implementation is less flak, at least in the local environment.

[github-actions]

	Fails
🚫	📋 Verify Changelog Format - See Changelog format

Generated by 🚫 dangerJS against 7c311bd

[samuelfreiberg]

Nit (optional recommendation, up to you): describe() should explain the feature as a whole being tested. Thinking about it from improving the debugging experience, I think the description should be:

"Testing Cancel Button Functionality"

This tells the engineer exactly which feature is under test.

[samuelfreiberg]

Nit: Improve test name. An engineer debugging this should know exactly what the test does by reading the name. After reading this, I'm not sure what this test is doing - I have to read the code to figure that out.

An example of a better name:

"Press Cancel Button - Validate {Correct Action}" but replace the {} with what you expect to happen

[samuelfreiberg]

I'm wondering if this should be in a before hook? Or it's own test? It looks like this is handling setting up the app to get the correct state, which doesn't correlate to the test name.

Making sure each test is has a defined function is important for debugging purposes (and code readability).

Rather than having the following structure:

it("Testing cancel button")
// Setup app
// Test Cancel Button

We can organize as:

it('Setup Cancel Button Testing')
// Setup app to test cancel button

it('Test cancel button')
// Test cancel button

[samuelfreiberg]

You can also include the error screenshots here. That way, the engineer can see a screenshot of the failure.

[samuelfreiberg]

Every test should have an assert (or else, what is it doing?).

Does this test have a purpose?

[samuelfreiberg]

Nit (Optional):

Before checking if the test passed, I added a small snippet of code that I found useful when debugging through logs. It can give you a quick method of searching for the results of a specific test in a long log file:

const resultString = results.passed ? 'Passed' : 'Failed';
console.log(``\nTest Case: "${test.description}".\nResult: "${resultString}".\nDuration: "${(results.duration/600).toFixed(2)}s". \n``);

[samuelfreiberg]

Nit: In all these helper methods, I suggest utilizing the errorMsg parameter in waitForDisplayed(). You can pass in the specific locator string so the error message is specific. If not, it will be confusing when debugging to see where it failed and which component.

For example,
await $(locator).waitForDisplayed({ timeout: 15000, timeoutMsg: "The element with locator=" + locator + " was not found"});

[samuelfreiberg]

It helps debugging a ton

[samuelfreiberg]

You can just return await $(locator).waitForDisplayed();. Such as return await $(locator).waitForDisplayed();.

waitForDisplayed throws an error if it fails, or returns true if it succeeds. Adding return $(locator).isDisplayed(); is unnecessary and will decrease perf by adding another API call.

[react-native-bot]

This PR is stale because it has been open 180 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[thymikee]

@Othinn any chance you could rebase that to main and migrate to GitHub Actions?