Discussion: One context per test, by default

[thernstig]

This is more of a discussion topic than a feature request/bug.

@pavelfeldman explained that the recommendation for Playwright is to use one context per test i.e. per it/test.

What are your thoughts about this? You do support this: https://github.com/playwright-community/jest-playwright#reset-current-context

My question is: Should this maybe be the default for jest-playwright going forward? I'd suspect that requires a new major release though.

[thernstig]

I find one big downside of this approach. Some apps can take some time to load on intial page load. Opening a new context per test, and thus one new page per test, can be slow in this scenario.

[mxschmitt]

I personally think this approach is the way to go that a Context per test -> it statement is the default behaviour.

[thernstig]

With the option to revert that behavior I suppose by some flag or something, for those who want one context per test file.

@mxschmitt what are your thoughts about page load speed? Recon it could be a problem, increasing test case runtime for users?

[mxschmitt]

I think its not a big deal, normally pages load in 0.5-2 seconds when they are served from localhost.

[thernstig]

In CI it might often be that you serve it in a cluster that is not on localhost. Yet I still find having like 2 seconds per test case quite a lot. 200 test cases is more than 3 minutes extra. Maybe I am just picky 😆

[mrienstra]

Documentation could be much more up-front about current behavior, it could be presented as something to grapple with early on.

[mrienstra]

jest-playwright.config.js could support a new reset option, something along the lines of:

reset: {
  file: 'browser',
  describe: 'persist',
  test: 'context'
}

... Or perhaps just:

reset: {
  describe: 'persist',
  test: 'context'
}

... as "per file" reset can to some degree be controlled with skipInitialization & launchType.

"Per describe" reset: I was imagining this would translate to "per nested describe" and ignore the top-level describe, the use case that springs to mind: describe wrapping a test.each, in which it is desirable to reset browser or context before / after the describe but not between each test.each test.