Description
Current behavior:
We have a spec file that intermittently fails partway through, after running several tests. While diagnosing, we found that this was due to a timeout of a cy.task
call. That call is only present in a before
for the top-level describe
, which already completed successfully.
When watching the video of the test failure, we saw that the page reloaded after running several tests, at the same point where the timeout occurred. Further investigation confirmed that when this reload occurred, the before
block was re-executed.
(Note: by "reloaded" I mean the page goes white and comes back, including the cypress portions of the page. The tests don't restart as they would if I manually refreshed the page.)
In our particular case, the cy.task
is performing database deletion and recreation in Cosmos DB, which can get cranky and throw 429 errors if it's called too frequently (not to mention even when it succeeds, deleting and recreating the database in an unexpected place in the tests is bound to cause problems).
We noticed that the place in the tests where the reload happens is the first place we call visit
on an actual web page--all the previous tests are only testing an API, so there is no need to load a webpage.
We added another before
block to visit
a page prior to our cy.task
to reset the database, and no longer see the page reload.
Desired behavior:
before
blocks should not be rerun partway through a test suite even if the tests trigger a page reload.
Steps to reproduce:
https://github.com/c32hedge/cypress-test-tiny/tree/5319
Observations from creating the MWE:
- The log statement in the second
it
executes twice--once before the console is cleared, and then again after thebefore
block runs again. - This behavior is only observed the first time the tests are run in a session (i.e. running with
cypress open
, if I click "Run all tests" again, the page doesn't reload and the log statements occur in the expected sequence.
Versions
Cypress 3.4.1
Ubuntu 18.04
Chrome 77.0.3865.90 (Official Build) (64-bit)