Retry-ability

[badeball]

Hi,

Retry-ability is a core feature of Cypress and part of what makes it so usable. Cypress also provides APIs to create custom commands that adhere to this pattern (using cy.should with functions, which is retried for you).

At my current project, we have some visual regression tests testing something with a very minor animation. We've successfully stored baseline images, but experience intermittent failures (0.3% - 0.5%) due to this animation.

I think it would be very cool for this library to do retry-ability, like we're all used to Cypress doing. I don't think it should be more than changing a .then with a .should, and using expect instead of throwing.

What do you think of this?

One challenge that does come to mind is when to store baseline images in scenarios with animations. This isn't however a new scenario, but something that is already difficult today as well.

[badeball]

Since 1) cy.screenshot is not retried by itself and 2) requires post-processing before comparison, this isn't as simple as using should and expect. You need to create a custom command and utilize cy.verifyUpcomingAssertions, as shown here.

[0xIslamTaha]

a workaround to achieve the same result is

cy.get('LOCATOR`).should('be.visible').matchImageSnapshot(snapshotName);

[Vandivier]

Hi @badeball would you mind reviewing this related issue? #118

We can consolidate support to one or the other issue. Let me know which you prefer. I think this issue implies it is a new feature request while the other issue implies the current docs indicate this should already be possible.

[badeball]

I don't have any strong preferences towards it.

Btw, I made a proof-of-concept of retry-ability here. Unforunately, I found the jest test to be highly coupled with implementation details and I'm hesitant in changing them (I think jest is the wrong way to test this and that acceptance tests in the form Cypress tests or tests writing tests is the only realistically good way of testing something like this).

[0xIslamTaha]

@badeball @Vandivier
could u check this PR #128 ?

[Vandivier]

@0xIslamTaha peer review is done.

[ChristophWalter]

Is there any progress on this? Or anything holding us back from implementing retry-ability?

[0xIslamTaha]

Is there any progress on this? Or anything holding us back from implementing retry-ability?

You could use my fork ✌️✌️

[lmeikle]

Cypress v5+ has retries in it.
We just need this PR to be merged as this library doesn't work with it yet :(
#155

[ChristophWalter]

In cypress there are two concepts: Retry-ability and (lately) Test Retries. Both have their purpose. But I am with you @lmeikle, saw your PR before and it should be merged/reviewed... Is this library still maintained?

[mauricedb]

I have been using this with patch-package for a while and it works just fine for me.