Repository navigation
Is playwright supported? Tests fail but show as passed #16
Description
The sdk branch appears to use framework: playwright, and this is GitHub repository is linked to in the setup documentation for Playwright and BrowserStack.
In the sdk branch, when you set retries: 3 in the playwright configuration file for re-running test failures and then run npm run sample-test command, (with your own user/access key) it marks the tests in BrowserStack as passed, but throws these errors in the console and the SDK marks the tests as failed, leading to it retrying over and over.
Running 4 tests using 2 workers
1) [playwright-firefox@latest:Windows 11@browserstack] › tests\bstack_test_add_to_cart.js:3:1 › BStackDemo test add to cart
Error: : expected object, got undefined
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: : expected object, got undefined
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
Error: : expected object, got undefined
Retry #3 ───────────────────────────────────────────────────────────────────────────────────────
Error: : expected object, got undefined
But in BrowserStack, it shows as passed sessions:
Activity
[-]Is playwright supported?[/-][+]Is playwright supported? Tests fail but show as passed[/+]i-love-code commentedon Dec 13, 2023
It would appear that the
browserstack-node-sdkthat is wrapping thenpx playwright testcommand does not correctly infer the outcome of the test, and marks all tests as having failed.i-love-code commentedon Dec 13, 2023
I have active BrowserStack support issues open for this, but I have not seen a helpful response/resolution. Trying all available paths for support.
rohit-y180 commentedon Dec 14, 2023
Hi @i-love-code , could you please let us know the support ticket ID that you have created on BrowserStack?