Performance issue and websocket error

[DavidZidar]

I just upgraded to version 1.0.0 and immediately noticed that my test suite takes much longer to run, now about 18 seconds when it finished in about 10 seconds before. My guess is there is something slow going on during startup because the first test result takes much longer to show up as done.

Even worse, every other time or so that I run my tests I get the following error during startup:

C:\path_to_project\node_modules\playwright-chromium\lib\server\chromium.js:211
            session.queue.push(parsedMessage);
                          ^

TypeError: Cannot read property 'push' of undefined
    at WebSocket.<anonymous> (C:\path_to_project\node_modules\playwright-chromium\lib\server\chromium.js:211:27)
    at WebSocket.emit (events.js:315:20)
    at Receiver.receiverOnMessage (C:\path_to_project\node_modules\ws\lib\websocket.js:789:20)
    at Receiver.emit (events.js:315:20)
    at Receiver.dataMessage (C:\path_to_project\node_modules\ws\lib\receiver.js:422:14)
    at Receiver.getData (C:\path_to_project\node_modules\ws\lib\receiver.js:352:17)
    at Receiver.startLoop (C:\path_to_project\node_modules\ws\lib\receiver.js:138:22)
    at Receiver._write (C:\path_to_project\node_modules\ws\lib\receiver.js:74:10)
    at writeOrBuffer (_stream_writable.js:352:12)
    at Receiver.Writable.write (_stream_writable.js:303:10)

This is not an error that I have seen before and when I revert to version 0.2.5 these issues go away.

[mxschmitt]

Hi @DavidZidar, which version of Playwright are you using? Playwright has afaik currently on the latest version with performance.

[mmarkelov]

@DavidZidar thank you for report. It will be nice if you could provide some more details. Your configuration, tests example, cause right now it's quite difficult to say what is wrong.
@mxschmitt what do you think about it? Seems like that problem could be in #134, #138 or #147

[DavidZidar]

@mxschmitt I'm using Playwright v1.1.1 both before and after, the only thing I changed was jest-playwright-preset version.

@mmarkelov Any specific thing you need to know about the configuration?

[mxschmitt]

@DavidZidar are you having multiple test files? multiple browsers / devices?
@mmarkelov coverage probably not, since its opt-in. Most likely the reuse browser change.

[DavidZidar]

Currently have 7 test files that Jest runs in parallell, they test two related websites running locally on my machine and I'm only using playwright-chromium.

[mmarkelov]

@DavidZidar can you try 1.0.0-rc1 version?
@mxschmitt if the issue will be reproduced in 1.0.0-rc1, that seems like something is wrong with reusing browser

[DavidZidar]

Just tried 1.0.0-rc1 and I'm still seeing the same issue

[DavidZidar]

I only see one chromium instance opening though.

Edit: I read too quickly,. Yes, it seems there may be an issue with reusing the browser if that's the only thing that changed in rc1.

[mmarkelov]

@mxschmitt maybe we can make reusing browser optional? Cause right now I don't have ideas how to fix it

[mxschmitt]

@mxschmitt maybe we can make reusing browser optional? Cause right now I don't have ideas how to fix it

I'll try to make a test to find out the cause for it instead of dropping the whole thing. Since its only a performance topic which adds a few seconds, I would only consider it as a medium issue instead of a super critical one.

[DavidZidar]

It is not only a performance issue, the tests completely fail to run when the error is thrown.

I see it happening mostly when I try running the tests twice in a row.

[mxschmitt]

@DavidZidar you are on Windows right? Thats maybe the issue, I can try to setup a VM and debug it myself. In the meantime it would be awesome if you could create a minimal example which would reproduce the issue.

[DavidZidar]

@mxschmitt That's right, I'm on Windows. I created this minimal repo https://github.com/DavidZidar/jest-playwright-preset-bug and can reproduce the issue fairly easily by running yarn test a couple of times.

While trying to reproduce the error this new error happened once, the last test never even finished, I aborted after a minute or so.

 PASS   browser: chromium  tests/two.test.js
 PASS   browser: chromium  tests/three.test.js
 PASS   browser: chromium  tests/four.test.js
 FAIL   browser: chromium  tests/one.test.js
  ● Test suite failed to run

    TimeoutError: Timeout 30000ms exceeded during chromium.connect.
    ================== chromium.connect logs ==================
    <ws connecting> ws://127.0.0.1:63171/46708a7182b95e66e568342e5eb20910
    <ws connected> ws://127.0.0.1:63171/46708a7182b95e66e568342e5eb20910
    ============================================================
    Note: use DEBUG=pw:api environment variable and rerun to capture Playwright logs.

      at ProgressController.run (node_modules/playwright-chromium/lib/progress.js:72:30) 

[mmarkelov]

@DavidZidar thank you for representation repo. I just tried to run it with Mac and it is OK. Seems like @mxschmitt is right, cause it maybe something that related to Windows

[DavidZidar]

You may be right. To be clear though the error doesn't happen every time, I tried it again and had to run it four times in a row before the error happened this time.