test: replace story screenshots with DOM snapshots #2619
Conversation
Excluding canvas based subsurface-viewer stories
…nts into test_runner/compose_dom_snapshots
…nts into test_runner/compose_dom_snapshots
typescript/packages/subsurface-viewer/src/storybook/components/DistanceScale.stories.tsx
Show resolved
Hide resolved
| Storybook test-runner is used to smoke test all stories in each package. | ||
|
|
||
| Tests can be run, given a running (`npm run storybook`) or static storybook, using `npm run docker:build && npm run docker:storybook:test`. | ||
| Tests can be run using `npm run compose:storybook:test` |
There was a problem hiding this comment.
Does this replace the previous version (ie. docker:build and docker:storybook:test), that is running in the docker ?
If so we can remove the docker:* version.
If not, add in the doc how to do it (needed for windows devs to update the ref images).
There was a problem hiding this comment.
I hoped everyone could use the new compose:* version. Could you give it a try?
DOM snapshots are not the same when running storybook dynamically. In CI it runs static, so you won't be able to create DOM snapshots locally with the previous scripts.
However, for creating screenshots, the dynamic version (docker:*) still works and is probably quicker.
There was a problem hiding this comment.
In that case, keep both version in the README and explain the difference.
There was a problem hiding this comment.
I deleted a reference .snap file and ran npm run compose:storybook:test => Result was green with a written snapshot :/
Snapshot Summary
› 1 snapshot written from 1 test suite.Test Suites: 45 passed, 45 total
Tests: 186 passed, 186 total
Snapshots: 1 written, 181 passed, 182 total
Time: 527.003 s
Ran all test suites.
Unfortunately, the EOL was not matching the initial file's EOL :/
A second run failed because of 2 image comparison failures...
Then I deleted a reference .snap file and ran npm run compose:storybook:test:update twice => it ended green, but without restoring the missing .snap file.
There was a problem hiding this comment.
I deleted a reference .snap file and ran
npm run compose:storybook:test=> Result was green with a written snapshot :/
Is this new behaviour?
Snapshot Summary
› 1 snapshot written from 1 test suite.
Test Suites: 45 passed, 45 total
Tests: 186 passed, 186 total
Snapshots: 1 written, 181 passed, 182 total
Time: 527.003 s
Ran all test suites.Unfortunately, the EOL was not matching the initial file's EOL :/
I don't understand how this could happen. Does it fail the tests?
A second run failed because of 2 image comparison failures...
Are the tests more flaky than before?
Then I deleted a reference .snap file and ran
npm run compose:storybook:test:updatetwice => it ended green, but without restoring the missing .snap file.
I haven't seen the update not working before. Can you show me more precisely what you did so that I can try to reproduce it?
| const stories: Meta = { | ||
| component: NumericInput, | ||
| title: "SubsurfaceViewer/Components/Settings", | ||
| tags: ["no-screenshot-test"], |
There was a problem hiding this comment.
Remove corresponding ref images
| const stories: Meta = { | ||
| component: ToggleButton, | ||
| title: "SubsurfaceViewer/Components/Settings", | ||
| tags: ["no-screenshot-test"], |
There was a problem hiding this comment.
Remove corresponding ref images
| Storybook test-runner is used to smoke test all stories in each package. | ||
|
|
||
| Tests can be run, given a running (`npm run storybook`) or static storybook, using `npm run docker:build && npm run docker:storybook:test`. | ||
| Tests can be run using `npm run compose:storybook:test` |
There was a problem hiding this comment.
In that case, keep both version in the README and explain the difference.
Replaced for some stories where possible and sensible.
Needed to switch to running tests on static Storybook due to inconsistencies between statically and dynamically generated DOM snapshots.