How to synchronise test with Pact to avoid flaky tests

[develmac]

So my guess is this is a reoccurring problem.
When you use any testing library for your WEB component tests you will have PactJS running on some port and waiting for interactions but the .verify() is going to be called from your unit test.

What is the best way to make sure that by the time you call .verify() all the expected network requests are already made (and have reached Pact)?
The best solution I could come up with so far is to return a promise from the component and wait on that in the test. This sometimes requires you to change your component code, what is imho not ideal. Maybe there is a wait with a timeout option in Pact? I could not find it in the docs. Not sure what waitForServerReady does?

Why this is relevant? Ofc you can make this sort of work, by adding a sleep or moving the verification into afterEach or similar. Regardless what you will get are flaky tests, that tend to break on different environments like CI/CD.

[mefellows]

Can you please elaborate perhaps with code examples of what you mean?

I've not seen this problem before, at least not framed the way you described.

Normally yes, you'd await the promise.

[develmac]

Sry, my bad! I have provided an example here: https://github.com/maconic/vue-vuetify-demo-project/blob/pact-sync/test/unit/example.test.ts

[mefellows]

Thanks for the example. Looking at your code, you seem to be testing at a much higher level (which I think you were implying with the "web component" bit in your descirption). This is not good practice / recommended (see item 3 of the "writing good consumer tests" guide).

My guess is that if you actually target the API client that is making the calls, you'll have more reliability in your tests, as you won't be waiting for click events and any subsequent API activity to complete.

[develmac]

I get the point, but sometimes the client is just generated code (for instance via OpenAPI generator) and the only thing that remains to interact with is the component. Still I guess it would be better to use the component as a white box and not do UI stuff? I am still a bit confused by the existence of the Cypress/Pact Plugin, isn't that then orthogonal to the guide you just linked?

[develmac]

To the point of writing good consumer tests, in the Pact-JVM guides I have also seen many examples where they spin up the whole component (for instance by using @SpringBootTest) and test the whole component. Isn't that also against this "rule" to test only the resource layer?

[YOU54F]

So it may framed this way, because pact-js is agnostic of the unit testing framework.

most unit testing frameworks provide a mechanism for setting a timeout, a set of maintainers built jest-pact and mocha-pact as wrappers around pact-js for that very function

https://github.com/pact-foundation/jest-pact
https://github.com/pact-foundation/mocha-pact

If you have synchronous code and don't want to introduce async/await, you can use callbacks.

This sometimes requires you to change your component code, what is imho not ideal

You may need to refactor code in order to make it testable, which is why testability of code is considered a good measure of code quality

[develmac]

Yes, the testing frameworks offer a way to synchronise, but the thing is that they just don't know if the Pact server already received any requests or not. This would mean that right now we would need to spam .verify() on pact in a try/catch. My thinking was maybe the verify call could have a timeout for the defined interaction to happen or not.