[Impeller] Add support for multi-rendering-backend fat shader archives. #47278

chinmaygarde · 2023-10-24T20:26:39Z

Existing shader archive ctors have been made private. Instead a factory is used to check to see if the blob is a single or multi-archive and do the right thing. This allows us to keep the existing code the same. But, when multi-archives are encountered, the right shaders libraries will be picked.

Creating multi-archives will be done in ImpellerC in an upcoming patch.

Towards flutter/flutter#123741

Existing shader archive ctors have been made private. Instead a factory is used to check to see if the blob is a single or multi-archive and do the right thing. This allows us to keep the existing code the same. But, when multi-archives are encountered, the right shaders libraries will be picked. Creating multi-archives will be done in ImpellerC in an upcoming patch. Towards flutter/flutter#123741

jonahwilliams

So I understand correctly, what we're doing here is adjusting the archival format so that it records shader data explicitly with a corresponding backend? And then later on, we can use this format to bundle both GLES/Vulkan?

jonahwilliams · 2023-10-24T22:32:24Z

impeller/shader_archive/multi_arch_shader_archive.fbs

+
+namespace impeller.fb;
+
+enum RenderingBackend:byte {


Should we include more information here? Right now there are some implicit version constraints in these formats more or less. We could make this explicitly kOpenGLESv100 or add additional version metadata.

You could imagine in the future that we might support targeting higher GLSL versions for desktop or webgl, and the engine will need to know the right version information to consume it. I'm not sure if we'd ever ship multiple GLSL versions of the same shader in the same file.

All of this is internal implementation detail. So I suspect we can tinker on this all we want.

In the future, we could even concatenate multiple shader libraries for the same backend instead of loading on of each.

jonahwilliams

LGTM with question

chinmaygarde · 2023-10-24T22:32:59Z

Right. The shaders can either be individual shader archives (like we have currently) or metallib files (for Metal only), or one of these multi-archives. Multi-archives just concatenate the blobs together with some metadata describing the blob.

zanderso · 2023-10-25T15:38:29Z

impeller/shader_archive/multi_arch_shader_archive.h

+      backend_mappings_;
+  bool is_valid_ = false;
+
+  FML_DISALLOW_COPY_AND_ASSIGN(MultiArchShaderArchive);


Are we still doing this in new code? Or should the new style be used?

Patched #47307

zanderso · 2023-10-25T15:38:58Z

impeller/shader_archive/multi_arch_shader_archive_writer.h

+  std::map<ArchiveRenderingBackend, std::shared_ptr<const fml::Mapping>>
+      archives_;
+
+  FML_DISALLOW_COPY_AND_ASSIGN(MultiArchShaderArchiveWriter);


Ditto question.

Really not convinced this is easier to read but maybe I just need to get used to it. Besides, good to just stick to the style guide uniformly. Followup from #47278 (comment) Can do this for the rest of the engine and remove the macros if this is fine.

…er archives. (flutter/engine#47278)

…137310) Roll Flutter Engine from 6e09ee14e244 to 6428ed576841 (38 revisions) flutter/engine@6e09ee1...6428ed5 2023-10-26 jason-simmons@users.noreply.github.com Revert Dart SDK to 360370ff93b053253343832432f8329a11372ffc (flutter/engine#47326) 2023-10-25 skia-flutter-autoroll@skia.org Roll Skia from fb979d0ce053 to a5da4435bb5c (1 revision) (flutter/engine#47325) 2023-10-25 jonahwilliams@google.com [Impeller] Cache location in metadata. (flutter/engine#46640) 2023-10-25 skia-flutter-autoroll@skia.org Roll Skia from 8492914821fa to fb979d0ce053 (1 revision) (flutter/engine#47324) 2023-10-25 737941+loic-sharma@users.noreply.github.com [Windows] Introduce FlutterWindowsViewController (flutter/engine#47242) 2023-10-25 skia-flutter-autoroll@skia.org Roll Skia from 773abacc1581 to 8492914821fa (2 revisions) (flutter/engine#47322) 2023-10-25 skia-flutter-autoroll@skia.org Roll Dart SDK from 73fa7b3e048a to 7745c140d1fe (1 revision) (flutter/engine#47321) 2023-10-25 zanderso@users.noreply.github.com Cleanup Dart package dependencies a bit (flutter/engine#47306) 2023-10-25 skia-flutter-autoroll@skia.org Roll Skia from d511fa1f1533 to 773abacc1581 (1 revision) (flutter/engine#47312) 2023-10-25 skia-flutter-autoroll@skia.org Roll Fuchsia Mac SDK from SOv1RJmbyDByvud8D... to kNdyn03p28H7VeMcd... (flutter/engine#47313) 2023-10-25 skia-flutter-autoroll@skia.org Roll Fuchsia Linux SDK from vX5n99OIWgLX6bObI... to zTq0jH2Y3Fl0uUUSa... (flutter/engine#47309) 2023-10-25 skia-flutter-autoroll@skia.org Roll Skia from 0748053419cd to d511fa1f1533 (1 revision) (flutter/engine#47308) 2023-10-25 30870216+gaaclarke@users.noreply.github.com Missed linter argument comment violations (flutter/engine#47310) 2023-10-25 chinmaygarde@google.com [Impeller] Remove use of FML_DISALLOW_<FOO> macros in Impeller. (flutter/engine#47307) 2023-10-25 30870216+gaaclarke@users.noreply.github.com Adds lint for checking argument commments (flutter/engine#47305) 2023-10-25 skia-flutter-autoroll@skia.org Roll Dart SDK from 2791495ed583 to 73fa7b3e048a (1 revision) (flutter/engine#47304) 2023-10-25 skia-flutter-autoroll@skia.org Roll Skia from 4848dd9b5dab to 0748053419cd (2 revisions) (flutter/engine#47302) 2023-10-25 skia-flutter-autoroll@skia.org Roll Skia from 788eafdcf70d to 4848dd9b5dab (1 revision) (flutter/engine#47301) 2023-10-25 skia-flutter-autoroll@skia.org Roll Skia from 5c315b761a24 to 788eafdcf70d (1 revision) (flutter/engine#47300) 2023-10-25 skia-flutter-autoroll@skia.org Roll Skia from 18ad006ec7f9 to 5c315b761a24 (1 revision) (flutter/engine#47299) 2023-10-25 skia-flutter-autoroll@skia.org Roll Dart SDK from 57661d5dbc1e to 2791495ed583 (1 revision) (flutter/engine#47298) 2023-10-25 skia-flutter-autoroll@skia.org Roll Skia from 9537c25509ea to 18ad006ec7f9 (1 revision) (flutter/engine#47297) 2023-10-25 skia-flutter-autoroll@skia.org Roll Fuchsia Mac SDK from YqSO1OByhoexFJSCr... to SOv1RJmbyDByvud8D... (flutter/engine#47296) 2023-10-25 skia-flutter-autoroll@skia.org Roll Skia from d51d3bcbcacd to 9537c25509ea (1 revision) (flutter/engine#47295) 2023-10-25 skia-flutter-autoroll@skia.org Roll Skia from f98fc9536c43 to d51d3bcbcacd (1 revision) (flutter/engine#47294) 2023-10-25 skia-flutter-autoroll@skia.org Roll Fuchsia Linux SDK from tqukMrdJ6j-845-57... to vX5n99OIWgLX6bObI... (flutter/engine#47293) 2023-10-25 skia-flutter-autoroll@skia.org Roll Skia from 0d4fcf388a6f to f98fc9536c43 (1 revision) (flutter/engine#47292) 2023-10-24 chinmaygarde@google.com [Impeller] Update shader compilation pipeline documentation to detail multi-arch archives. (flutter/engine#47289) 2023-10-24 skia-flutter-autoroll@skia.org Roll Skia from 076a9dd0094f to 0d4fcf388a6f (2 revisions) (flutter/engine#47290) 2023-10-24 matanlurey@users.noreply.github.com OpenGL <4.x does not support stencil-only formats, so delete the TODO (flutter/engine#47286) 2023-10-24 matanlurey@users.noreply.github.com [Impeller] Unconditionally dither in gradient shader fragments. (flutter/engine#46746) 2023-10-24 skia-flutter-autoroll@skia.org Roll Skia from f491209e969d to 076a9dd0094f (1 revision) (flutter/engine#47285) 2023-10-24 chinmaygarde@google.com [Impeller] Add support for multi-rendering-backend fat shader archives. (flutter/engine#47278) 2023-10-24 skia-flutter-autoroll@skia.org Roll Skia from 502277be15cf to f491209e969d (1 revision) (flutter/engine#47281) 2023-10-24 skia-flutter-autoroll@skia.org Roll Dart SDK from 360370ff93b0 to 57661d5dbc1e (1 revision) (flutter/engine#47280) 2023-10-24 30870216+gaaclarke@users.noreply.github.com Migrated away from UnmodifiableUint8ListView (flutter/engine#47276) 2023-10-24 skia-flutter-autoroll@skia.org Roll Skia from bc90585b0dd4 to 502277be15cf (1 revision) (flutter/engine#47277) 2023-10-24 1961493+harryterkelsen@users.noreply.github.com Reland "Use a single OffscreenCanvas for rendering in CanvasKit (#45744)" (flutter/engine#47241) Also rolling transitive DEPS: fuchsia/sdk/core/linux-amd64 from tqukMrdJ6j-8 to zTq0jH2Y3Fl0 fuchsia/sdk/core/mac-amd64 from YqSO1OByhoex to kNdyn03p28H7 If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll ...

Fixes breakages from: * flutter/engine#47080 * flutter/engine#47247 * flutter/engine#47278

bdero/impeller-cmake#20 Fixes breakages from: * #47080 * #47247 * #47278

@bdero

This is part of the work towards supporting OpenGLES and Vulkan for runtime stage shaders. Removes some redundant work we had around SkSL. Now only bundles the shaders we actually ask for from the command line. @bdero, we should figure out if this is the right approach for flutter_gpu. With this change, the IPLR format goes from having a root table of shader related information to a root table of shader information per `sksl`, `metal`, `opengles`, and `vulkan` platforms. This may end up allowing us to revert #47278, but I'm not sure I understand all the implications of that at this point. I have run some but not all tests locally.

chinmaygarde requested review from bdero, zanderso and jonahwilliams October 24, 2023 20:26

github-actions bot added the e: impeller label Oct 24, 2023

Licenses.

e60f61e

jonahwilliams reviewed Oct 24, 2023

View reviewed changes

jonahwilliams approved these changes Oct 24, 2023

View reviewed changes

chinmaygarde added the autosubmit Merge PR when tree becomes green via auto submit App label Oct 24, 2023

auto-submit bot merged commit 502755d into flutter:main Oct 24, 2023

chinmaygarde deleted the multi_arch branch October 24, 2023 22:38

zanderso reviewed Oct 25, 2023

View reviewed changes

chinmaygarde mentioned this pull request Oct 25, 2023

[Impeller] Remove use of FML_DISALLOW_<FOO> macros in Impeller. #47307

Merged

engine-flutter-autoroll mentioned this pull request Oct 25, 2023

Roll Flutter Engine from 6e09ee14e244 to 53a8d4de211f (28 revisions) flutter/flutter#137286

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Oct 25, 2023

502755d68 [Impeller] Add support for multi-rendering-backend fat shad…

3735520

…er archives. (flutter/engine#47278)

engine-flutter-autoroll mentioned this pull request Oct 25, 2023

Roll Flutter Engine from 6e09ee14e244 to e88745051c6e (31 revisions) flutter/flutter#137291

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Oct 25, 2023

502755d68 [Impeller] Add support for multi-rendering-backend fat shad…

d65e648

…er archives. (flutter/engine#47278)

engine-flutter-autoroll mentioned this pull request Oct 25, 2023

Roll Flutter Engine from 6e09ee14e244 to c18f8f1b9e7c (33 revisions) flutter/flutter#137296

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Oct 25, 2023

502755d68 [Impeller] Add support for multi-rendering-backend fat shad…

fe45635

…er archives. (flutter/engine#47278)

engine-flutter-autoroll mentioned this pull request Oct 25, 2023

Roll Flutter Engine from 6e09ee14e244 to 6de8127713bc (36 revisions) flutter/flutter#137300

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Oct 25, 2023

502755d68 [Impeller] Add support for multi-rendering-backend fat shad…

47e5ad2

…er archives. (flutter/engine#47278)

engine-flutter-autoroll mentioned this pull request Oct 26, 2023

Roll Flutter Engine from 6e09ee14e244 to 236562dacd71 (37 revisions) flutter/flutter#137305

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Oct 26, 2023

502755d68 [Impeller] Add support for multi-rendering-backend fat shad…

14ca211

…er archives. (flutter/engine#47278)

engine-flutter-autoroll mentioned this pull request Oct 26, 2023

Roll Flutter Engine from 6e09ee14e244 to 6428ed576841 (38 revisions) flutter/flutter#137310

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Oct 26, 2023

502755d68 [Impeller] Add support for multi-rendering-backend fat shad…

6bff166

…er archives. (flutter/engine#47278)

bdero added a commit to bdero/impeller-cmake that referenced this pull request Nov 1, 2023

Bump Flutter Engine to head.

2678b56

Fixes breakages from: * flutter/engine#47080 * flutter/engine#47247 * flutter/engine#47278

bdero mentioned this pull request Nov 1, 2023

Bump Flutter Engine to head. bdero/impeller-cmake#20

Merged

bdero added a commit to bdero/impeller-cmake that referenced this pull request Nov 1, 2023

Bump Flutter Engine to head.

4ea085d

Fixes breakages from: * flutter/engine#47080 * flutter/engine#47247 * flutter/engine#47278

bdero added a commit to bdero/impeller-cmake that referenced this pull request Nov 1, 2023

Bump Flutter Engine to head.

ffb5582

Fixes breakages from: * flutter/engine#47080 * flutter/engine#47247 * flutter/engine#47278

bdero mentioned this pull request Nov 1, 2023

Bump impeller-cmake to head #47563

Merged

bdero added a commit that referenced this pull request Nov 1, 2023

Bump impeller-cmake to head (#47563)

c896f5f

bdero/impeller-cmake#20 Fixes breakages from: * #47080 * #47247 * #47278

dnfield mentioned this pull request Dec 19, 2023

[Impeller] Make IPLR files multi-platform #49253

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Impeller] Add support for multi-rendering-backend fat shader archives. #47278

[Impeller] Add support for multi-rendering-backend fat shader archives. #47278

Uh oh!

chinmaygarde commented Oct 24, 2023

Uh oh!

jonahwilliams left a comment

Uh oh!

jonahwilliams Oct 24, 2023

Uh oh!

chinmaygarde Oct 24, 2023

Uh oh!

jonahwilliams Oct 24, 2023

Uh oh!

jonahwilliams left a comment

Uh oh!

chinmaygarde commented Oct 24, 2023

Uh oh!

zanderso Oct 25, 2023

Uh oh!

chinmaygarde Oct 25, 2023 •

edited

Loading

Uh oh!

zanderso Oct 25, 2023

Uh oh!

Uh oh!


		namespace impeller.fb;

		enum RenderingBackend:byte {

[Impeller] Add support for multi-rendering-backend fat shader archives. #47278

[Impeller] Add support for multi-rendering-backend fat shader archives. #47278

Uh oh!

Conversation

chinmaygarde commented Oct 24, 2023

Uh oh!

jonahwilliams left a comment

Choose a reason for hiding this comment

Uh oh!

jonahwilliams Oct 24, 2023

Choose a reason for hiding this comment

Uh oh!

chinmaygarde Oct 24, 2023

Choose a reason for hiding this comment

Uh oh!

jonahwilliams Oct 24, 2023

Choose a reason for hiding this comment

Uh oh!

jonahwilliams left a comment

Choose a reason for hiding this comment

Uh oh!

chinmaygarde commented Oct 24, 2023

Uh oh!

zanderso Oct 25, 2023

Choose a reason for hiding this comment

Uh oh!

chinmaygarde Oct 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zanderso Oct 25, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

chinmaygarde Oct 25, 2023 •

edited

Loading