Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

focus SkiaGPUObject wrappers on only those objects that need the protection #41237

Merged
merged 3 commits into from
Apr 17, 2023

Conversation

flar
Copy link
Contributor

@flar flar commented Apr 15, 2023

With this PR we no longer need to hold DisplayLists in GPUObject wrappers and they can be disposed instantly instead of queueing on the Unref thread.

This will definitely be a win for Impeller as none of the objects used in a frame now require queueing, but the performance impact on apps running on top of skia is less clear if they depend on a lot of images inside their DisplayLists that still need to be queued to be freed. After getting further in the work, it looks like only decoded images need to use the protected DlImage wrappers and most of those should survive many frames before they are disposed. That should hopefully leave very few unrefs happening per frame.

There are 3 unit tests in shell_unittests.cc and embedder_metal_unittests.mm that are now GSKIP'd as they now invoke code that needs a fully initialized UIDartState in order to protect their images. I will look into fixing the tests and/or making the code they invoke provide protection without relying on UIDartState. (This looks to be fixed in the latest commit by simply not creating DlImageGPUs all over the source base and simply catching only those that end up in UI data structures. There is actually existing code in one of the modules that feeds ui.Image with an answer to wrap the image in a DlImageGPU if it has a skia image anyway, so most of these additional uses of DlImageGPU that were having trouble getting the Skia unref queue just didn't need it anyway.)

@flutter-dashboard flutter-dashboard bot added embedder Related to the embedder API platform-android labels Apr 15, 2023
@flar flar force-pushed the DL-no-GPUObject-wrapper branch from be430f9 to d82cdb5 Compare April 15, 2023 06:27
@flar
Copy link
Contributor Author

flar commented Apr 15, 2023

It looks like there are more unit tests that run into the same problem.

I'm thinking the right fix for that is to find a way for the code in shell/ to get access to the SkiaUnrefQueue and make its own SkiaGPUObject wrappers without having to rely on the UIDartState code in lib/ui... It turns out that nearly all of these can just return a DlImage and either their image is not destined to be sent to the UI thread anyway, or there is a guard in the lib/ui code that checks if an image is a Skia image and makes sure it is wrapped before adding it to a UI data structure.

@flar flar force-pushed the DL-no-GPUObject-wrapper branch from d82cdb5 to 6b47eba Compare April 17, 2023 21:49
@flar
Copy link
Contributor Author

flar commented Apr 17, 2023

The use of SkiaGPUObject wrappers is now limited to the following files that need to wrap an image for use in a UI thread:

% git grep skia_gpu_object     
ci/licenses_golden/ ...
flow/BUILD.gn:    "skia_gpu_object.h",
flow/BUILD.gn:      "skia_gpu_object_unittests.cc",
flow/skia_gpu_object_unittests.cc:#include "flutter/flow/skia_gpu_object.h"
lib/ui/io_manager.h:#include "flutter/flow/skia_gpu_object.h"
lib/ui/painting/display_list_image_gpu.h:#include "flutter/flow/skia_gpu_object.h"
shell/common/shell_io_manager.h:#include "flutter/flow/skia_gpu_object.h"
% git grep SkiaGPUObject  
flow/skia_gpu_object.h: ...
flow/skia_gpu_object_unittests.cc: ...
lib/ui/painting/display_list_image_gpu.cc:sk_sp<DlImageGPU> DlImageGPU::Make(SkiaGPUObject<SkImage> image) {
lib/ui/painting/display_list_image_gpu.cc:DlImageGPU::DlImageGPU(SkiaGPUObject<SkImage> image)
lib/ui/painting/display_list_image_gpu.h:  static sk_sp<DlImageGPU> Make(SkiaGPUObject<SkImage> image);
lib/ui/painting/display_list_image_gpu.h:  SkiaGPUObject<SkImage> image_;
lib/ui/painting/display_list_image_gpu.h:  explicit DlImageGPU(SkiaGPUObject<SkImage> image);
lib/ui/painting/image_decoder_skia.cc:static SkiaGPUObject<SkImage> UploadRasterImage(
lib/ui/painting/image_decoder_skia.cc:  SkiaGPUObject<SkImage> result;
lib/ui/painting/image_decoder_skia.cc:          SkiaGPUObject<SkImage> image, fml::tracing::TraceFlow flow) {

@flar
Copy link
Contributor Author

flar commented Apr 17, 2023

And only a couple of files need to use the enqueueing DlImageGPU as seen here:

% git grep DlImageGPU
lib/ui/painting/display_list_image_gpu.cc: ...
lib/ui/painting/display_list_image_gpu.h: ...
lib/ui/painting/image_decoder_skia.cc:              callback(DlImageGPU::Make(std::move(image)));
lib/ui/painting/multi_frame_codec.cc:  return DlImageGPU::Make({skImage, std::move(unref_queue)});
lib/ui/painting/picture.cc:              DlImageGPU::Make({image->skia_image(), std::move(unref_queue)});
shell/common/snapshot_controller.h:  // be converted to a DlImageGPU if it is to be handed back to the UI
shell/common/snapshot_controller_skia.cc:  // It is up to the caller to create a DlImageGPU version of this image

Note that the last 2 references outside of lib/ui are just comments that those lines of code are not responsible for protecting the images (and, in fact, the code in picture.cc that you see here is doing just that on their behalf).

@flar flar added the autosubmit Merge PR when tree becomes green via auto submit App label Apr 17, 2023
@auto-submit auto-submit bot merged commit 35833e7 into flutter:main Apr 17, 2023
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Apr 18, 2023
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Apr 18, 2023
zanderso pushed a commit to flutter/flutter that referenced this pull request Apr 18, 2023
…125039)

flutter/engine@c4396f9...d297361

2023-04-18 leroux_bruno@yahoo.fr [macOS] Fix Ctrl+Tab is broken
(flutter/engine#40706)
2023-04-18 skia-flutter-autoroll@skia.org Roll Fuchsia Mac SDK from
K1LGtKXyxRlW3Q9O1... to qZHSvkpAU1-YYGvYc... (flutter/engine#41293)
2023-04-18 aam@google.com Roll buildroot to
059d155b4d452efd9c4427c45cddfd9445144869 (flutter/engine#41288)
2023-04-18 bdero@google.com [Impeller] Remove glyph pixel rounding
during text frame conversion (flutter/engine#41285)
2023-04-18 godofredoc@google.com Revert "Reland "Migrate mac_host_engine
to engine v2 builds."" (flutter/engine#41284)
2023-04-17 flar@google.com focus SkiaGPUObject wrappers on only those
objects that need the protection (flutter/engine#41237)

Also rolling transitive DEPS:
  fuchsia/sdk/core/mac-amd64 from K1LGtKXyxRlW to qZHSvkpAU1-Y

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC chinmaygarde@google.com,rmistry@google.com,zra@google.com on
the revert to ensure that a human
is aware of the problem.

To file a bug in Flutter:
https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://bugs.chromium.org/p/skia/issues/entry?template=Autoroller+Bug

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
autosubmit Merge PR when tree becomes green via auto submit App embedder Related to the embedder API platform-android
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants