[Impeller] read from framebuffer for advanced blends on iOS.

[jonahwilliams]

Work towards flutter/flutter#120223 for iOS. This dramatically improves performance of the pull quote on wonderous by reducing the number of subpasses and removing the required full screen fill. from 8ms down to 2-3 ms, vs 1.5-2ms for Skia. So pretty fast, and with less memory too

Note that this excludes the shaders from non iOS platforms but does not exclude the normal advanced blend shaders from iOS. Since we're not doing this in filter contents I've decided to punt on whether or not we should refactor that to take advantage of this - speciifically for drawVertices/drawAtlas there would be more followup work

I also left out support for a foreground color, but we should probably add that back and find some way to detect offscreen solid color contents. Or just always convert SolidColorContents into a 1x1 texture of that solid color when calling RenderToSnapshot.

Here is a fun test app for this change:

import 'package:flutter/material.dart';

void main() {
  runApp(const MaterialApp(home: Scaffold(body: Example())));
}

class Example extends StatefulWidget {
  const Example({super.key});

  @override
  State<Example> createState() => _ExampleState();
}

class _ExampleState extends State<Example> {
  var hellos = [
    'Hello',
  ];

  @override
  Widget build(BuildContext context) {
    return Stack(
      children: [
          Container(color: Colors.grey),
          GridView.count(crossAxisCount: 8, children: [
            for (int i = 0; i < 100; i++)
              Text('  ${hellos[i % hellos.length]}  ', style: TextStyle(fontSize: 40))
          ]),
          GridView.count(crossAxisCount: 8, children: [
            for (int i = 0; i < 200; i++)
              CustomPaint(
                painter: FooPainter(i),
                size: Size(50, 50),
              ),
          ]),
      ]
    );
  }
}

class FooPainter extends CustomPainter {
  FooPainter(this.i);

  final int i;

  var colors = [
    Colors.red,
    Colors.green,
    Colors.blue,
    Colors.yellow,
    Colors.purple,
    Colors.orange,
    Colors.lime,
  ];

  @override
  void paint(Canvas canvas, Size size) {
    canvas.drawRect(Rect.fromLTWH(0, 0, size.width, size.height), Paint()..color = colors[i % colors.length] .. blendMode = BlendMode.screen);
  }

  @override
  bool shouldRepaint(covariant CustomPainter oldDelegate) {
    return true;
  }
}

[jonahwilliams]

Here is the colordodge shader compiled (abbreviated):

#include <metal_stdlib>
#include <simd/simd.h>

using namespace metal;

struct BlendInfo
{
    float dst_input_alpha;
    float src_y_coord_scale;
    float color_factor;
    float4 color;
};

struct framebuffer_blend_colordodge_fragment_main_out
{
    float4 frag_color [[color(0)]];
};

struct framebuffer_blend_colordodge_fragment_main_in
{
    float2 v_src_texture_coords [[user(locn0)]];
};

fragment framebuffer_blend_colordodge_fragment_main_out framebuffer_blend_colordodge_fragment_main(framebuffer_blend_colordodge_fragment_main_in in [[stage_in]], constant BlendInfo& blend_info [[buffer(0)]], float4 uSub [[color(0)]], texture2d<float> texture_sampler_src [[texture(0)]], sampler texture_sampler_srcSmplr [[sampler(0)]])
{
    framebuffer_blend_colordodge_fragment_main_out out = {};
    float4 _380 = uSub;
    ...
    return out;
}

Of course this is resulting in solid black output. note I did leave some deb

[jonahwilliams]

This now works on simple examples but fails on wonderous. Probably some coverage/transform/stencil issue I haven't shaken out yet.

Also note that I made an effort to exclude the framebuffer shaders on non ios platforms but did not make an effort to remove the regular advanced blends. TBD if we want to do that.

[jonahwilliams]

Note to self: remember to remove

[jonahwilliams]

@bdero WDUT about the approximate approach here?

(Also checking if you can spot the bug before I do 🦅 )

[jonahwilliams]

frame debugger has the geometry in the right place, and it worked for simpler examples. i made the coverage a huge rect. 🤔

[bdero]

Overall approach LGTM! 👍 I glanced through and didn't spot the geometry bug. Is stuff getting partially cut off or getting culled away completely? Do the UV values reasonable in the frame debugger?

[jonahwilliams]

The draw call itself isn't getting culled - I can see the geometry and uvs look reasonable. Actually changing it to output a solid color results in no differences in wonderous so I don't think its a UV issue anyway

Not sure what else to look at. Very simple examples seem to work fine.

[jonahwilliams]

Wait, am I actually adding this correctly or am I actually just replacing the last entity.

[jonahwilliams]

Well I guess yes, since I call renderToSnapshot on the contents I replace.

[jonahwilliams]

ahh, just forgot to set the stencil 🤦

[flutter-dashboard]

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption to this rule, contact Hixie on the #hackers channel in Chat (don't just cc him here, he won't see it! He's on Discord!).

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

[jonahwilliams]

Looks like I also need to disable this on iOS simulators.

[bdero]

LGTM!

[bdero]

So reading from the framebuffer wouldn't happen to work on Apple silicon macs, would it? :)

[jonahwilliams]

it might be, but from the errors I hit it sounds like it needs a substantially higher MSL version:

https://github.com/KhronosGroup/SPIRV-Cross/blob/4e2fdb25671c742a9fbe93a6034eb1542244c7e1/spirv_msl.cpp#L13082

[bdero]

Oh, well that turned out simpler than I was expecting!