ADR 009: Guidelines for interaction tests in primer-react #2223
Conversation
|
size-limit report 📦
|
|
Thanks for contributing this ADR, @pksjce! In order to help move a decision forward, I'm assigning the following: ADR decision maker: @colebemis Your role as decision maker is to gather feedback, probe for discussion and disagreement, and then make an informed decision based on the feedback, identified risks, and trade-offs about whether to adopt the ADR or not. Your role is not to decide based on your own personally preferred approach. Keep in mind that ADR decisions can be reversed or changed in the future if new information is exposed that make the ADR worth revisiting. You can fulfill your decision maker role by doing the following:
|
| Our testing infrastructure consists of Jest "unit" tests and automatic Chromatic visual testing. Both of these are super useful but we have been experiencing a few setbacks. | ||
| Unofficially we test more complex interactive scenarios using our storybooks setup. | ||
|
|
||
| 1. Repeated markup in storybooks and jest tests - Since there is not automation of interaction in storybooks, we tend to test them out manually in stories and then transfer to jest. This results in us constructing similar test case scenarios in storybooks and in tests. |
There was a problem hiding this comment.
10/10 can confirm. Feel like a caveman doing this.
| Going forward, we can use [storybook's play functions](https://storybook.js.org/docs/react/writing-stories/play-function) to write interactive tests for our components. Here's a document that describes h[ow to add and run these tests](../storybook-tests.md). | ||
| Guidelines to make this easier - | ||
|
|
||
| 1. Create a new storybook file for your components and call them "interactions". Inside these files, we can import the stories from "examples" or "fixtures" and tack on play functions to them. This will create duplicate stories but with the added benefit of the tests appearing in the interactions add-on tab. This tab will let you step through your tests step by step, increasing visibility and debuggability. |
There was a problem hiding this comment.
Thanks for sharing this @siddharthkp 🙌🏼
There was a problem hiding this comment.
@siddharthkp - Do you think I should link this stuff in the ADR?
There was a problem hiding this comment.
Approved! from my side
I like the approach a lot more than writing interactive tests that run with jest. My only fear (probably irrational?) with this approach is that by going deeper into the storybook ecosystem, we might get locked in with the features they do or don't provide?
(The other visual alternative would be to use playwright which run a chromium instance that can be intercepted to debug (memex uses this approach), but the dev experience is not as smooth as the one in @pksjce's demo)
Finally, an unsolved problem that we'll need to figure out: How do we add results from these tests to our coverage metrics?
|
Thanks so much for putting this ADR together @pksjce ! It helped me understand where we are at testing and what our options are. I haven't used Storybook's interaction tests before but I like the approach. Are we thinking them as an alternative to E2E testing or are we thinking starting with Storybook play functions and down the road, we can introduce a more comprehensive tool for E2E and integrate into our CI/CD? |
|
|
||
| ## Decisions | ||
|
|
||
| Going forward, we can use [storybook's play functions](https://storybook.js.org/docs/react/writing-stories/play-function) to write interactive tests for our components. Here's a document that describes h[ow to add and run these tests](../storybook-tests.md). |
There was a problem hiding this comment.
| Going forward, we can use [storybook's play functions](https://storybook.js.org/docs/react/writing-stories/play-function) to write interactive tests for our components. Here's a document that describes h[ow to add and run these tests](../storybook-tests.md). | |
| Going forward, we can use [storybook's play functions](https://storybook.js.org/docs/react/writing-stories/play-function) to write interactive tests for our components. Here's a document that describes [how to add and run these tests](../storybook-tests.md). |
Typo :)
|
|
||
| 2. Port complex jest tests onto interactive tests. This is up to developer discretion. My recommendation is to have basic "unit" tests like component existence and axe tests in jest. Anything that can benefit from a visual aspect to it can be moved to interaction tests. | ||
|
|
||
| 3. Interaction tests are not perfect. Some aspects like mocking can be challenging and we will find solutions to these as when the need arises. |
There was a problem hiding this comment.
Out of curiosity, did we consider any alternative options to Storybook play functions?
If so, it'd be nice to add a section about what else we considered. If not, all good.
There was a problem hiding this comment.
This is a good idea. I did have thoughts around cypress and playwright before this. Not sure how much in detail I should mention them. Will add to this draft.
|
@siddharthkp - Thanks so much for your comments. Re- Deep storybook integration. Re- Test Coverage - I've been looking into this since the last time you mentioned it in chat. I'm having a slightly hard time coming up with a solution for this. Maybe we could pair sometime next week and brainstorm what to do for this? |
pksjce
force-pushed
the
pk/adr-interaction-tests
branch
from
ce73102
to
b9ca708
Compare
Yep, I'm pretty clueless as well, let's do that :) |
Describe your changes here.
This ADR proposes some instructions for how we should write interaction tests. Feedback welcome!