[ci] Run web tests in wasm (unit + integration). #8111

ditman · 2024-11-16T00:35:23Z

Adds CI configuration to run web integration tests (in the master channel) compiled to Wasm.

It also removes the build-examples step from web integration tests, in some isolated testing:

platform tests shard	With build-examples	Without build-examples
1	30m	21m
2	13m	11m
3	17m	10m

Issues

Fixes Add packages repo CI job to run web integration tests compiled to WASM flutter#151664

Pre-launch Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

ditman

Changes are required, do not merge as is.

.ci/targets/web_platform_tests.yaml

.ci.yaml

ditman · 2024-12-20T03:51:56Z

packages/pointer_interceptor/pointer_interceptor_web/example/integration_test/widget_test.dart

+  // Give the browser some time to perform DOM operations (for Wasm code)
+  await tester.pump(const Duration(milliseconds: 500));


Talk to @eyebrowsoffire about this new delay.

We discussed this, the skwasm renderer is now async, so it makes sense that there's some timing differences compared to the JS one. Needing this additional delay is probably caused by the tester not being aware of this asynchronicity, and declaring a frame "pumped" before it's really been fully rendered.

packages/url_launcher/url_launcher_web/example/integration_test/link_widget_test.dart

packages/video_player/video_player/example/integration_test/video_player_test.dart

…ion to test CI before it propagates.

… wasm in all_apps

… different entrypoint each time; I'm not sure prebuilding helps with anything other than pre-fetching deps (?)

ditman · 2024-12-24T02:54:15Z

[0:00] Running for packages/animations...
  No version change.
  Found NEXT; validating next version in the CHANGELOG.
No CHANGELOG change found.
If this PR needs an exemption from the standard policy of listing all changes in the CHANGELOG,
comment in the PR to explain why the PR is exempt, and add (or ask your reviewer to add) the
"override: no changelog needed" label.
Otherwise, please add a NEXT entry in the CHANGELOG as described in the contributing guide.

The only change to packages/animations is the .gitignore, which is having a flutter fix that I introduced in May 2022 (here). No need to version anything, it's purely repo related.

ditman · 2025-01-06T19:38:41Z

packages/vector_graphics_codec/lib/vector_graphics_codec.dart

-    if (focalX != null) {
+    if (focalX != null && focalY != null) {


See: dart-lang/sdk#59840

stuartmorgan-g · 2025-01-06T19:43:31Z

My only question is if we want to run the tests both on master and stable or master only is fine.

My rule of thumb for expensive tests is to ask: is someone likely to make a change that only breaks on the combination of stable and the new config? Or can we get almost as much confidence that if some PR passes some configuration on stable, and also passes both configurations on master, that it'll be fine on the other configuration on stable?

If we don't have that confidence, then we can add (ideally post-submit-only) stable versions, otherwise I tend to lean toward reducing CI load.

stuartmorgan-g

LGTM

packages/animations/example/.gitignore

ditman · 2025-01-07T01:56:52Z

[...] otherwise I tend to lean toward reducing CI load.

@stuartmorgan I agree, let's go with just master for now. I'm not sure what can be introduced into wasm compilation that would break this stuff too badly only on stable (but we're very smart people, I'm sure we can find a way :P)

Anyway, it's easy enough to add the stable shards down the line, so let's go with this!

ditman · 2025-01-07T18:40:35Z

The following packages had errors:
  packages/animations:
    Missing version change

Adding label to skip version change in packages/animations. The only change is in the .gitignore file (see message above)

ditman · 2025-01-07T19:18:11Z

One last update to give flakes a chance to reconsider their failing stance :P

…ackages#8111)

flutter/packages@3fc6b7a...6554751 2025-01-09 stanleycocoa@gmail.com [webview_flutter_wkwebview] Add a listener for the canGoBack property change on the iOS platform. (flutter/packages#8203) 2025-01-09 39571861+memishood@users.noreply.github.com [flutter_svg] Implement errorBuilder callback (flutter/packages#8364) 2025-01-08 10687576+bparrishMines@users.noreply.github.com [interactive_media_ads] Adds remaining methods for internal wrapper of the iOS native `IMAAdDisplayContainer`. (flutter/packages#8153) 2025-01-08 jhy03261997@gmail.com [go_router_builder] Add a fallback for a not null List or Set param (flutter/packages#8349) 2025-01-08 engine-flutter-autoroll@skia.org Manual roll Flutter (stable) from 8495dee to 17025dd (2 revisions) (flutter/packages#8396) 2025-01-08 10687576+bparrishMines@users.noreply.github.com [ci][video_player_android][camera_android_camerax] Manual roll and fix tests (flutter/packages#8391) 2025-01-07 ditman@gmail.com [ci] Run web tests in wasm (unit + integration). (flutter/packages#8111) 2025-01-07 liff@iki.fi [camera_platform_interface] Add API support query for image streaming (flutter/packages#8307) 2025-01-07 10687576+bparrishMines@users.noreply.github.com [camera_android_camerax][video_player_android] Suppresses deprecation of the `SurfaceProducer.Callback.onSurfaceDestroyed` method (flutter/packages#8388) 2025-01-07 jessiewong401@gmail.com Bump Remaining Plugin Example Apps to targetSdkVersion >= 34 and compileSdkVersion >= 34 (flutter/packages#8361) 2025-01-07 matanlurey@users.noreply.github.com Replace `onSurfaceCreated` with `onSurfaceAvailable`. (flutter/packages#8384) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages-flutter-autoroll Please CC flutter-ecosystem@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

This PR moves the new Wasm shards out of bringup, so they become part of the normal testing flow. Here's some information about the last ~90 builds in the staging builder: * Unit tests, master channel * [Shard 1](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_dart_unit_test_wasm_shard_1%20master?limit=200) * [Shard 2](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_dart_unit_test_wasm_shard_2%20master?limit=200) * Integration tests, master channel * [Shard 1](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_1%20master?limit=200) * [Shard 2](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_2%20master?limit=200) * [Shard 3](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_3%20master?limit=200) ## Issues * Continuation of #8111 * Completes flutter/flutter#151664 * Closes flutter/flutter#127730

Adds CI configuration to run web integration tests (in the master channel) compiled to Wasm. It also removes the `build-examples` step from web integration tests, in some isolated testing: | platform tests shard | With build-examples | Without build-examples | |---|-----|-----| | 1 | 30m | 21m | | 2 | 13m | 11m | | 3 | 17m | 10m | ## Issues * Fixes flutter/flutter#151664

This PR moves the new Wasm shards out of bringup, so they become part of the normal testing flow. Here's some information about the last ~90 builds in the staging builder: * Unit tests, master channel * [Shard 1](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_dart_unit_test_wasm_shard_1%20master?limit=200) * [Shard 2](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_dart_unit_test_wasm_shard_2%20master?limit=200) * Integration tests, master channel * [Shard 1](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_1%20master?limit=200) * [Shard 2](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_2%20master?limit=200) * [Shard 3](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_3%20master?limit=200) ## Issues * Continuation of flutter#8111 * Completes flutter/flutter#151664 * Closes flutter/flutter#127730

Adds CI configuration to run web integration tests (in the master channel) compiled to Wasm. It also removes the `build-examples` step from web integration tests, in some isolated testing: | platform tests shard | With build-examples | Without build-examples | |---|-----|-----| | 1 | 30m | 21m | | 2 | 13m | 11m | | 3 | 17m | 10m | ## Issues * Fixes flutter/flutter#151664

This PR moves the new Wasm shards out of bringup, so they become part of the normal testing flow. Here's some information about the last ~90 builds in the staging builder: * Unit tests, master channel * [Shard 1](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_dart_unit_test_wasm_shard_1%20master?limit=200) * [Shard 2](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_dart_unit_test_wasm_shard_2%20master?limit=200) * Integration tests, master channel * [Shard 1](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_1%20master?limit=200) * [Shard 2](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_2%20master?limit=200) * [Shard 3](https://ci.chromium.org/ui/p/flutter/builders/staging/Linux_web%20web_platform_tests_wasm_shard_3%20master?limit=200) ## Issues * Continuation of flutter#8111 * Completes flutter/flutter#151664 * Closes flutter/flutter#127730

ditman force-pushed the run-web-integration-wasm branch from 8a28941 to d6bdc4f Compare December 17, 2024 23:35

github-actions bot added p: pointer_interceptor platform-web p: video_player p: url_launcher p: google_maps_flutter labels Dec 20, 2024

ditman commented Dec 20, 2024

View reviewed changes

github-actions bot added the p: animations label Dec 20, 2024

ditman added 16 commits December 23, 2024 15:28

Add a --wasm test shard in master.

a1d0b38

Add web_platform_tests_wasm config. Copy that config to non-wasm vers…

9f4cd1f

…ion to test CI before it propagates.

Remove extra comma

b518877

url_launcher_web tests now pass with --wasm

0f80fcb

video_player tests now pass with --wasm

1542405

pointer_interceptor_web wasm tests now pass.

1685463

Exclude file_selector, since it does not have integration tests.

b56ac8d

Attempt to pass multiple exclusion files, similarly to how we exclude…

c510654

… wasm in all_apps

google_maps_flutter_web tests now pass in wasm

bcd7f6f

dart format .

01a9321

Remove unnecessary import.

6b01a26

Clarify note

be17b43

Shard wasm tests in 3 shards.

9c10258

Try prebuilding the examples.

8bb9e64

Do not prebuild apps. Integration tests on the web work by building a…

18674b3

… different entrypoint each time; I'm not sure prebuilding helps with anything other than pre-fetching deps (?)

Update animations/example gitignore file.

598ed80

ditman force-pushed the run-web-integration-wasm branch from 420958b to 598ed80 Compare December 23, 2024 23:28

ditman added the override: no changelog needed Override the check requiring CHANGELOG updates for most changes label Dec 24, 2024

ditman added 3 commits December 23, 2024 19:41

Merge branch 'main' into run-web-integration-wasm

630bfe6

Merge branch 'flutter:main' into run-web-integration-wasm

dc40e4d

google_adsense Fix experimental test so it passes in wasm.

67d7897

ditman requested review from jonahwilliams and Hixie as code owners January 6, 2025 19:33

ditman requested a review from eyebrowsoffire January 6, 2025 19:33

ditman commented Jan 6, 2025

View reviewed changes

stuartmorgan-g approved these changes Jan 6, 2025

View reviewed changes

packages/animations/example/.gitignore Outdated Show resolved Hide resolved

Remove unneeded comment from gitignore

6660cfb

ditman added the override: no versioning needed Override the check requiring version bumps for most changes label Jan 7, 2025

ditman changed the title ~~[ci][web] Run web tests in wasm (unit + integration).~~ [ci] Run web tests in wasm (unit + integration). Jan 7, 2025

Merge branch 'main' into run-web-integration-wasm

fcaff2a

ditman added the autosubmit Merge PR when tree becomes green via auto submit App label Jan 7, 2025

auto-submit bot merged commit 11a9fa8 into flutter:main Jan 7, 2025
78 checks passed

engine-flutter-autoroll mentioned this pull request Jan 8, 2025

Roll Packages from 3fc6b7ace3ff to 11a9fa8bf6e5 (5 revisions) flutter/flutter#161315

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Jan 8, 2025

11a9fa8bf [ci] Run web tests in wasm (unit + integration). (flutter/p…

e61c372

…ackages#8111)

engine-flutter-autoroll mentioned this pull request Jan 8, 2025

Roll Packages from 3fc6b7ace3ff to 11a9fa8bf6e5 (5 revisions) flutter/flutter#161325

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Jan 8, 2025

11a9fa8bf [ci] Run web tests in wasm (unit + integration). (flutter/p…

888b5ea

…ackages#8111)

engine-flutter-autoroll mentioned this pull request Jan 9, 2025

Roll Packages from 3fc6b7ace3ff to 65547511c004 (11 revisions) flutter/flutter#161379

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Jan 9, 2025

11a9fa8bf [ci] Run web tests in wasm (unit + integration). (flutter/p…

e71b756

…ackages#8111)

ditman mentioned this pull request Jan 16, 2025

[google_maps_flutter] Support for Ground Overlay #8432

Merged

11 tasks

This was referenced Feb 5, 2025

[ci] Move Wasm test shards out of bringup. #8572

Merged

[integration_test] Flutter Web WASM support flutter/flutter#127730

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[ci] Run web tests in wasm (unit + integration). #8111

[ci] Run web tests in wasm (unit + integration). #8111

Uh oh!

ditman commented Nov 16, 2024 •

edited

Loading

Uh oh!

ditman left a comment

Uh oh!

Uh oh!

Uh oh!

ditman Dec 20, 2024

Uh oh!

ditman Dec 20, 2024

Uh oh!

Uh oh!

Uh oh!

ditman commented Dec 24, 2024

Uh oh!

ditman Jan 6, 2025

Uh oh!

stuartmorgan-g commented Jan 6, 2025

Uh oh!

stuartmorgan-g left a comment

Uh oh!

Uh oh!

ditman commented Jan 7, 2025

Uh oh!

ditman commented Jan 7, 2025 •

edited

Loading

Uh oh!

ditman commented Jan 7, 2025

Uh oh!

Uh oh!

Uh oh!

		// Give the browser some time to perform DOM operations (for Wasm code)
		await tester.pump(const Duration(milliseconds: 500));

		if (focalX != null) {
		if (focalX != null && focalY != null) {

[ci] Run web tests in wasm (unit + integration). #8111

[ci] Run web tests in wasm (unit + integration). #8111

Uh oh!

Conversation

ditman commented Nov 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issues

Pre-launch Checklist

Uh oh!

ditman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ditman Dec 20, 2024

Choose a reason for hiding this comment

Uh oh!

ditman Dec 20, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ditman commented Dec 24, 2024

Uh oh!

ditman Jan 6, 2025

Choose a reason for hiding this comment

Uh oh!

stuartmorgan-g commented Jan 6, 2025

Uh oh!

stuartmorgan-g left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ditman commented Jan 7, 2025

Uh oh!

ditman commented Jan 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ditman commented Jan 7, 2025

Uh oh!

Uh oh!

Uh oh!

ditman commented Nov 16, 2024 •

edited

Loading

ditman commented Jan 7, 2025 •

edited

Loading