How to use additional flags to a 'jest-playwright --parallel' script?

[MtthsB]

Dear contributors

I have recently switched from puppeteer to playwright, but I'm struggling to implement a - what I feel should be - simple use case. I would like to avoid running a multiple and parallel cross-browser test on all my tests, and instead just focus on those test files that have changed, cf. the --o flag in jest. Furthermore, I only want to use playwright for the virtual regression tests, not for purely functional unit tests. I have created separate files for both types of tests, and would therefore discern between the two by passing a specific --testRegex flag.

I am using the following setup:
"jest": "^24.8.0",
"jest-playwright-preset": "^0.1.3",
"jest-screenshot": "^0.2.2",
"playwright": "^1.0.1"
... and I have implemented the recommended jest-playwright.config.js to enable multiple browser testing and I have set the jest setup in the package json as indicated.

As it stands, I can run the "jest-playwright --parallel --testRegex='"(.)*\.vr\.test\.js(x)"'" command and it will smoothly find the test files named ***.vr.test.jsx and test them. However, if I want to add ANYTHING to that command (such as the --o to watch only changed files, or to update a screenshot with --update), it will fail and run multiple browser tests for everything that matches the default jest testMatch regexes.

I don't know what I'm missing, but I've traced the settings as far down as I could and can only find that for some reason the testRegex is reverted to an empty array in the playwrightEnvironment instance and it is literally impossible to work with the testing framework.

I would love to contribute with a solution, but I'm afraid I got stuck debugging... Any help would be awesome.

thanks in advance!

[dgozman]

cc @mmarkelov

[mxschmitt]

Hi @MtthsB,
we recently rewrote the handling of multiple browsers, so would be awesome if you could try 0.2.0 (0.2.1, latest has probably a bug) of jest-playwright.
Do you want to only run the tests of the changed files based on local Git repository?
I don't get it fully what you are trying to archive.

[MtthsB]

Hi @mxschmitt

Thanks for the information. I'll upgrade to 0.2.0, and see what happens. I'm fully aware of the stage this package is in, so I appreciate any help without expecting it ;) I would love to contribute, but, as I've said, it's a bit out of my league atm.

Regardless, I just want a script that runs a test for all 3 browsers in parallel but can be modified with additional (jest-style) flags, such as --testRegex (the files that need to be tested with playwright are suffixed with .vr.test.jsx), --o (only the test files that have changed relative to the Git repo) or --update (in case the tests/snapshots need updating).

I hope this clarifies things somewhat. Like I said, running 'jest-playwright --parallel --testRegex <insert some regex here>'' already runs the parallel multi-browser tests, but if I try to add additional script flags to it, it seems to drop all flags and just runs 'jest-playwright --parallel'.

[mmarkelov]

@MtthsB there was quite hacky solution with parallel tag for previous versions of jest-playwright. So we just rewrote logic for supporting running tests for multiple browsers.
With 0.2.0 you should able to specify this with jest-playwright.config.js:

module.exports = {
    browsers: ["chromium", "webkit", "firefox"],
    ...
}

If it doesn't work as expected, fill free to open issue here, and I'll try to help

[MtthsB]

Hi @mmarkelov

I upgraded to 0.2.0. I had been using the jest-playwright.config.js settings as indicated on the jest-playwright-preset site.
When using 0.1.x I had to run the script with 'jest-playwright' and '--parallel', which I did. It worked, to some extent => additional flags were mostly ignored, which reduced the usability of the package considerably (hence the original question).

Now, with 0.2.0, I have reverted the command 'jest-playwright' to the normal 'jest'. However, it crashes with the error TypeError: browsers.flatMap is not a function at Promise.all.tests.map (/Users/saplingmats/Code/DP/focus-design-system/node_modules/jest-playwright-preset/lib/PlaywrightRunner.js:37:25) -- thought you might want to know. In other words, it seems less stable, if anything.

script: 'jest --testRegex='"(.)*\.vr\.test\.js(x)"''
jest-playwright.config.js => module.exports = {
launchBrowserApp: {
dumpio: true,
headless: true,
args: ['--no-sandbox', '--disable-gpu'],
},
browsers: ['chromium', 'firefox', 'webkit'],
};

Interestingly, just running 'jest', despite the preset in the config set to 'jest-playwright-preset', the test does not run multiple browser tests with those settings in jest-playwright-config.js

[mmarkelov]

@MtthsB as I told earlier jest-playwright 0.1 had a lot of bugs, especially with passing jest flags. So, the new version tries to fix it and make testing more stable and easier. It's not perfect -we are working on it. As I told, you should open issue in jest-playwright repo.
I suppose crash came from node: flatMap was introduced in Node 11. So it seems that you are using some older version.

[MtthsB]

@mmarkelov Thanks, it seems to be much stabler now. I've been able to make it run (after updating my node version) smoothly.

Keep up the great work!