Add playwright smoke tests for storybook #5329

dac09 · 2022-04-25T09:16:23Z

Closes #5276

What does the PR do?

Adds playwright smoke tests, after launching storybook on the test project.

There are 3 tests:

Cell mocks are loaded correctly
Cell mocks are also loaded when nested in a page (blogpostpage)
mockCurrentUser does what it says on the tin

And implicitly - that the storybook command, and server are working as expected

Video walk through

(sorry for the bad audio)

netlify · 2022-04-25T09:16:30Z

✅ Deploy Preview for redwoodjs-docs canceled.

Name	Link
🔨 Latest commit	`859a6c3`
🔍 Latest deploy log	https://app.netlify.com/sites/redwoodjs-docs/deploys/62713bcbdceeed0008310024

virtuoushub · 2022-04-25T14:02:12Z

Awesome start!

Couple of questions:

Do we want this to test the build as well as the dev server? I imagine the tasks/smoke-test/tests/storybook.spec.ts can be reused and we'd just want to modify/fork tasks/smoke-test/playwright-fixtures/storybook.fixture.ts to also do a build. I see value in both testing the dev server, and the built artifact.
What is the plan for the legacy test: https://github.com/redwoodjs/redwood/blob/main/tasks/e2e/cypress/integration/03-storybook/storybook.spec.js ? iirc, it is currently disabled; do we have plans on removing it altogether?

dac09 · 2022-04-27T06:11:17Z

Do we want this to test the build as well as the dev server? I imagine the tasks/smoke-test/tests/storybook.spec.ts can be reused and we'd just want to modify/fork tasks/smoke-test/playwright-fixtures/storybook.fixture.ts to also do a build. I see value in both testing the dev server, and the built artifact.

Yeah good point - I think certainly something we can add a test for, but I don't think we have a clear picture on how to even deploy the storybook after a build at the moment. Very happy to start looking at this option, and once we have documentation and an understanding of how we think it should work, we can add a smoke test

What is the plan for the legacy test: https://github.com/redwoodjs/redwood/blob/main/tasks/e2e/cypress/integration/03-storybook/storybook.spec.js ? iirc, it is currently disabled; do we have plans on removing it altogether?

I'll remove it!

experiment with loaders the loaders are async and execute before the story is rendered https://storybook.js.org/docs/react/writing-stories/loaders https://mswjs.io/ Co-authored-by: Yann Braga <yannbf@gmail.com>

…ok-smoke-test * 'main' of github.com:redwoodjs/redwood: Use GraphQL Codegen to generate more accurate resolver types (redwoodjs#5216) chore(deps): update dependency @clerk/clerk-sdk-node to v3.3.8 (redwoodjs#5345) fix(deps): update dependency jest-watch-typeahead to v1.1.0 (redwoodjs#5339)

…test

dac09 · 2022-05-02T02:08:05Z

Ok @virtuoushub - so I've been trying super hard to get these smoke tests to pass on Linux and Windows - and it looks like our currently implementation of mockCurrentUser without the loader, can be flakey under CI/Codespaces. Under normal human use I think ti doesn't matter, because by the time we refresh the page the mocks become available, but when playwright is running the test, maybe its too fast for the mocks to load?

I suspect this has something to do with:
a) The distro
b) Memory availability since e2e tests are heavy

I tried merging in your code in #4919 and the tests seem to pass - which says to me that the loader implementation is more solid!

~~Shall we merge that in then and merge in these tests after? Or maybe i'll just pull in your changes to this branch?~~
I've merged your code into this branch

…uoushub/redwood into feat/storybook-smoke-test

virtuoushub · 2022-05-02T14:21:14Z

... and it looks like our currently implementation of mockCurrentUser without the loader, can be flakey under CI/Codespaces.
...
I tried merging in your code in #4919 and the tests seem to pass - which says to me that the loader implementation is more solid!

Awesome! And yes, that PR is directly meant to address the some of the issues you noticed in CI/CD. Glad you are able to take advantage.

… and it has not loaded

…ok-smoke-test * 'main' of github.com:redwoodjs/redwood: Remove extra checkout in RC workflow (redwoodjs#5414) chore(deps): update dependency @azure/msal-browser to v2.24.0 (redwoodjs#5412) fix(deps): update react monorepo (redwoodjs#5406) cli upgrade: Always search from the start for semvers (redwoodjs#5368) codegen graphql schema (redwoodjs#5213) fix(deps): update dependency core-js to v3.22.4 (redwoodjs#5409) fix(deps): update typescript-eslint monorepo to v5.22.0 (redwoodjs#5410) Add graphql-scalars to graphql-server (redwoodjs#5408) Update yarn.lock v1.2.1 fix(deps): update dependency cross-undici-fetch to v0.3.6 (redwoodjs#5402) fix(deps): update dependency cross-undici-fetch to v0.3.5 (redwoodjs#5398) fix(deps): update dependency cross-undici-fetch to v0.3.3 (redwoodjs#5378) chore(story📗): extract MSW logic into a loader (redwoodjs#4919)

…into feat/storybook-smoke-test * 'feat/storybook-smoke-test' of github.com:dac09/redwood:

This reverts commit 343c156.

dac09 · 2022-05-03T03:34:48Z

FINALLY. Tests passing on all platforms. Ready for final review.

virtuoushub

🙇🏻‍♂️

dac09 added 4 commits April 25, 2022 13:46

Get it in working order

13f04a5

Revert launch to execa instead of sync

3e976ad

Optimise sb fixture

688129c

Add mockCurrentUser storybook smoke test

f769d57

jtoar assigned dac09 Apr 25, 2022

dac09 requested a review from virtuoushub April 25, 2022 09:16

dac09 added the release:chore This PR is a chore (means nothing for users) label Apr 25, 2022

dac09 added 2 commits April 25, 2022 16:31

Update title on ci workflow

e4fe164

Merge branch 'main' into feat/storybook-smoke-test

4dc6bd7

jtoar assigned virtuoushub Apr 25, 2022

dac09 added 2 commits April 27, 2022 13:05

Update fixture and spec for windows

7fb2560

Merge branch 'main' into feat/storybook-smoke-test

680648d

dac09 and others added 15 commits April 27, 2022 13:12

Remove outdated storybook spec in e2e

f9b74cb

Reload page after change to story

f485cb4

chore(story📗): extract MSW logic into a loader

dad672d

experiment with loaders the loaders are async and execute before the story is rendered https://storybook.js.org/docs/react/writing-stories/loaders https://mswjs.io/ Co-authored-by: Yann Braga <yannbf@gmail.com>

Try increasing timeout for windows

4b5a3e8

Set large timeout for all platforms

31f8e2d

Reuse existing storybook server

e03e628

Wait for load state after changing src

367d032

try reloading too

73ec476

Try different method of waiting for hotreload

79436ae

Longer timeouts for Windows

c846c5c

Try to debug failures

1097183

Only upload artefacts on failure

b35bf32

Merge branch 'chore/use-storybook-loaders' into feat/storybook-smoke-…

367e9d1

…test

Merge branch 'main' into chore/use-storybook-loaders

238cb43

dac09 added 3 commits May 2, 2022 08:56

Move story modification back to inside the test

0830ad3

Merge branch 'main' into feat/storybook-smoke-test

1a9690e

Merge branch 'main' into chore/use-storybook-loaders

a7d6877

dac09 added 2 commits May 2, 2022 05:37

Merge branch 'chore/use-storybook-loaders' of https://github.com/virt…

ffb59fe

…uoushub/redwood into feat/storybook-smoke-test

Merge branch 'main' into feat/storybook-smoke-test

907eee9

virtuoushub mentioned this pull request May 2, 2022

chore(story📗): extract MSW logic into a loader #4919

Merged

dac09 added 11 commits May 2, 2022 21:24

Merge branch 'main' into feat/storybook-smoke-test

40cb62c

Add a reload before profilepage checks

9de17b2

Dont modify src during tests

343c156

Mock current user part of test-project

282b04c

Merge branch 'main' into feat/storybook-smoke-test

0210c8b

Reduce timeouts | Add a refresh when waiting for mocked current user,…

c0e1916

… and it has not loaded

Merge branch 'feat/storybook-smoke-test' of github.com:dac09/redwood …

a1b8781

…into feat/storybook-smoke-test * 'feat/storybook-smoke-test' of github.com:dac09/redwood:

Revert "Dont modify src during tests"

2d28cf3

This reverts commit 343c156.

Still modify in test

ae7b38b

Use correct selector for profile

c46ee9d

dac09 added 2 commits May 3, 2022 10:36

Remove test-project generator modification of story

c61f79c

Merge branch 'main' into feat/storybook-smoke-test

432e741

virtuoushub approved these changes May 3, 2022

View reviewed changes

Merge branch 'main' into feat/storybook-smoke-test

859a6c3

dac09 enabled auto-merge (squash) May 3, 2022 14:27

dac09 merged commit 4129692 into redwoodjs:main May 3, 2022

jtoar added this to the next-release milestone May 3, 2022

dac09 deleted the feat/storybook-smoke-test branch May 3, 2022 14:54

jtoar modified the milestones: next-release, v1.4.0 May 5, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add playwright smoke tests for storybook #5329

Add playwright smoke tests for storybook #5329

dac09 commented Apr 25, 2022 •

edited

Loading

netlify bot commented Apr 25, 2022 •

edited

Loading

virtuoushub commented Apr 25, 2022 •

edited

Loading

dac09 commented Apr 27, 2022

dac09 commented May 2, 2022 •

edited

Loading

virtuoushub commented May 2, 2022

dac09 commented May 3, 2022

virtuoushub left a comment

Add playwright smoke tests for storybook #5329

Add playwright smoke tests for storybook #5329

Conversation

dac09 commented Apr 25, 2022 • edited Loading

What does the PR do?

Video walk through

netlify bot commented Apr 25, 2022 • edited Loading

✅ Deploy Preview for redwoodjs-docs canceled.

virtuoushub commented Apr 25, 2022 • edited Loading

dac09 commented Apr 27, 2022

dac09 commented May 2, 2022 • edited Loading

virtuoushub commented May 2, 2022

dac09 commented May 3, 2022

virtuoushub left a comment

Choose a reason for hiding this comment

dac09 commented Apr 25, 2022 •

edited

Loading

netlify bot commented Apr 25, 2022 •

edited

Loading

virtuoushub commented Apr 25, 2022 •

edited

Loading

dac09 commented May 2, 2022 •

edited

Loading