Golden comparison fails when pumping more than two times

[adigladi]

Description

We've been noticing golden errors in our project when using a rive animation. The error occurs if the test widget is being pumped three times (or more) in a test. As soon as three pumps are used, the golden comparison including a rive animation gets a very small diff where some pixels are being rendered different. This is blocking our UI testing since we can't get consistent results from the rive animation. We've tested on different rive animations with the same inconsistent result.

Steps To Reproduce

Steps to reproduce the behavior:

1. Run a golden test including any rive animation, e.g:

testWidgets('Rive animation matches golden', (WidgetTester tester) async {
      await tester.pumpWidget(
        const RiveAnimation.asset('rive.riv'),
      );
      // Pumping 2 times works, 3 times does not.
      await tester.pump();
      await tester.pump();
      await tester.pump();

      await expectLater(
          find.byType(RiveAnimation), matchesGoldenFile('goldens/rive_animation.png'));
    });

3. Generate golden by running flutter test --update-goldens
4. Run test again without updating golden flutter test
5. See golden error diff

Source .riv/.rev file

This seems to happen for any riv used, but here's one downloaded from the community which gives this error:

Cat_Rive.zip

Expected behavior

The golden test comparison should be consistent when using several pumps. It should be the same when updating the golden using 3 pumps and then running the same test using 3 pumps. Currently it is not.

Screenshots

Test image (using our internal rive since it is a bit more clear on the diff):

Golden failure isolated diff

Device & Versions

• Device: [Flutter Test]
• OS: [M1 Mac OSX 13.0.1]
• Flutter Version:

Flutter 3.7.6 • channel stable • https://github.com/flutter/flutter.git
Framework • revision 12cb4eb7a0 (5 weeks ago) • 2023-03-01 10:29:26 -0800
Engine • revision ada363ee93
Tools • Dart 2.19.3 • DevTools 2.20.1

Additional context

This could be related to the ticker issue described in #187

[HayesGordon]

Hi @adigladi,

Thanks for opening the issue. This should now be resolved in v0.11.14

It was indeed related to the ticker issue you linked.

Going to close this now, but please reopen if the issue persists.

[adigladi]

Thanks for fixing this! 🎉 🙌