cy.session: Caching across spec files

[halfninja]

What would you like?

The new cy.session is amazing and just what we wanted - the behaviour with cypress open where this feature caches across spec files is something we'd like to have as an option for cypress run as well if possible.

Why is this needed?

In many cases authentication sessions are reliably the same between spec files, and starting with a blank cy.session cache for each file isn't necessary. I understand that the reason may be more technical, since the runner may be opening each spec file in a brand new context where it's hard to carry session data around.

Other

Just further thanks for cy.session! 🎉

[IanVS]

This would be incredibly useful for my auth flow, which involves a magic link being sent to an email box and two-factor authentication. The auth takes quite a while to complete, and it would be awesome to cache the result across specs.

[bahmutov]

You could do this using cypress-data-session plugin https://github.com/bahmutov/cypress-data-session#videos

[henry-filosa]

I created a PR to advance discussion on a possible implementation of this. I'd also advocate for possibly removing the cache clearing behavior from run mode as it can be exactly replicated by invoking Cypress.session.clearAllSavedSessions()
in a before:spec hook. However, there is currently no option to disable.

I looked at the closed roadmap for the session feature, but it differs a fair bit from what we currently have. Is there a planning document for where the team is expecting to take the API of sessions in the future?

[romankhomitskyi]

I created a PR to advance discussion on a possible implementation of this. I'd also advocate for possibly removing the cache clearing behavior from run mode as it can be exactly replicated by invoking Cypress.session.clearAllSavedSessions() in a before:spec hook. However, there is currently no option to disable.

I looked at the closed roadmap for the session feature, but it differs a fair bit from what we currently have. Is there a planning document for where the team is expecting to take the API of sessions in the future?

Just curious, wouldn't it be better just add option not to clear local storage, cookies rather than creating huge API that is difficult to pick up?

[henry-filosa]

@romankhomitskyi You'll probably addressing this question at the Cypress team, but I'd like to answer as someone who's excited about this development. It brings local storage and cookies in line with the Cypress principle of setting up state before each test. This ensures greater test isolation for less flake.
Plus, it's already an optional flag so if you don't like it you can just not enable. However, it doesn't make sense to cache cookies if you're not clearing them.

[romankhomitskyi]

@romankhomitskyi You'll probably addressing this question at the Cypress team, but I'd like to answer as someone who's excited about this development. It brings local storage and cookies in line with the Cypress principle of setting up state before each test. This ensures greater test isolation for less flake. Plus, it's already an optional flag so if you don't like it you can just not enable. However, it doesn't make sense to cache cookies if you're not clearing them.

I am just wondering what kind of flake could be in my flow, where I created bunch of it() to check only one page, and after first it() I get logged out
As it was said before, most of the time you do not want to clear local storage
There might be a flake in cy.session down the road, that might not be fixed asap, so we can't rely on it(

[henry-filosa]

Those are questions that are really only answerable by you and your unique application. However, a "bunch of it() to check only one page" is probably not following best practices. But again, Cypress is flexible enough for you to follow the workflow you described.

I think a larger discussion about the merits of the API would be more appropriate in another forum.

[henry-filosa]

In future versions the behavior of Cypress in open mode will match that of run mode.. It seems this desired behavior is out of line with where the API is going. For those interested in pursuing such behavior I can confirm that @bahmutov's plugin provides a solution.

However, it seems inadvisable if you may want parallelization at some point for your project, which can deliver large speedups. When parallelized, spec files are routed nondeterministically to different virtual machines. It's not possible (at least as far as I can see) to share state between these machines, therefore each spec file should be written as if it could run in any combination of other spec files running. For our part, we have decided to look back at our architecture to find a different solution that works within this limitation.

[IanVS]

I understand the benefits of parallelization and test isolation, but shouldn't there be some kind of way to set up and store auth state? Especially for complex auth flows that involve sending and checking emails?

[henry-filosa]

The response on the PR:

Does your solution work when running tests in parallel across several machines? If not, then this will introduce non-determinism when running tests in parallel versus not. This is not behavior we want to introduce unless it behaves the same across all cypress run situations.

It seems a state sharing solution will have to work in parallel mode for it to make it into the Cypress project. Bahmutov's plugin seems like a good solution if that's not a concern for you.

[IanVS]

Ideally, there would be a way to run a login spec first, cache the session, and then use it across the rest of the tests in parallel.