[Impeller] Use a squircle-sdf-based algorithm for fast blurs #55604

flar · 2024-10-03T03:25:34Z

Use a Squircle Signed Distance Field based algorithm to do a very fast approximation of rrect blurs.

This PR is to provide reference within the team to discuss the future of the algorithm compared to the Gaussian approximation functions that are currently in use. It isn't a complete solution, but can be completed easily with a little more work.

Notably, it doesn't handle elliptical round rects, only circular corners.
Could stand to include an attribution to the source (https://raphlinus.github.io/graphics/2020/04/21/blurred-rounded-rects.html)

flutter-dashboard · 2024-10-03T03:25:36Z

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption, contact "@test-exemption-reviewer" in the #hackers channel in Discord (don't just cc them here, they won't see it!).

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. The test exemption team is a small volunteer group, so all reviewers should feel empowered to ask for tests, without delegating that responsibility entirely to the test exemption group.

gaaclarke · 2024-10-03T17:24:27Z

The falloff looks a bit abrupt with the current math. I'm working on getting the side-by-side to verify.

gaaclarke · 2024-10-03T18:25:21Z

Here is the golden changes: https://flutter-engine-gold.skia.org/search?issue=55604&crs=github&patchsets=2&corpus=flutter-engine. It looks really close. Here is one of the tests that show the oval not working though (see green shapes):

gaaclarke · 2024-10-03T18:38:06Z

Here is an example of some of the artifacts this removes:

gaaclarke · 2024-10-03T19:58:04Z

I was chatting with jonah in office. He was saying for ovals we should maybe just fall back to the full gaussian blur and drop the old rrect_blur completely (instead of keeping it around for ovals).

gaaclarke · 2024-10-03T20:30:10Z

I've came up with a test that shows what we need to fix to get this landed. There is the known issue of the oval which we knew about needs to fall back to regular blur, but also looks like small sigmas aren't rendered. We can probably just clamp that to 1:

TEST_P(AiksTest, SolidColorCirclesOvalsRRectsMaskBlurTinySigma) {
  auto callback = [&]() -> sk_sp<DisplayList> {
    if (AiksTest::ImGuiBegin("Controls", nullptr,
                             ImGuiWindowFlags_AlwaysAutoResize)) {
      ImGui::End();
    }
    DisplayListBuilder builder;
    builder.Scale(GetContentScale().x, GetContentScale().y);

    std::vector<float> sigmas = {0.0, 0.01, 1.0};
    std::vector<DlColor> colors = {DlColor::kGreen(), DlColor::kYellow(),
                                   DlColor::kRed()};
    for (uint32_t i = 0; i < sigmas.size(); ++i) {
      DlPaint paint;
      paint.setColor(colors[i]);
      paint.setMaskFilter(
          DlBlurMaskFilter::Make(DlBlurStyle::kNormal, sigmas[i]));

      builder.Save();
      builder.Translate(100 + (i * 100), 100);
      SkRRect rrect =
          SkRRect::MakeRectXY(SkRect::MakeXYWH(0, 0, 60.0f, 160.0f), 50, 100);
      builder.DrawRRect(rrect, paint);
      builder.Restore();
    }

    return builder.Build();
  };

  ASSERT_TRUE(OpenPlaygroundHere(callback));
}

expected output

seen output

gaaclarke · 2024-10-03T20:40:54Z

@flar I can't remember where you asked but we do have a rrect blur benchmark in flutter/dev/benchmarks/macrobenchmarks, it was added here: flutter/flutter#149261

gaaclarke · 2024-10-03T22:48:21Z

There are artifacts in the benchmark too, this is probably the same issue we see above with the small sigma.

sweepartifact.mov

gaaclarke · 2024-10-03T23:12:10Z

The render problem seems to happen around sigma=0.57 and I believe it is because of frag_info.scale == 0. That's calculated here:

  frag_info.scale =
      0.5 * computeErf7(frag_info.sInv * 0.5 *
                        (std::max(rSize.x, rSize.y) - 0.5 * radius));

In computeErf7, at the sqrt x ~ 2e19 so x * x = inf.

static Scalar computeErf7(Scalar x) {
  x *= kTwoOverSqrtPi;
  float xx = x * x;
  x = x + (0.24295 + (0.03395 + 0.0104 * xx) * xx) * (x * xx);
  return x / sqrt(1.0 + x * x);
}

gaaclarke · 2024-10-04T00:39:50Z

Here's the measurements:

before

after

skia

gaaclarke · 2024-10-04T00:40:34Z

Funny thing though is there is a bug in skia like the one I had to find for rendering small sigmas.

gaaclarke · 2024-10-04T16:34:47Z

Here is the regular gaussian blur for reference. It looks about the same gpu-wise with the current rrect_blur, but the gaussian blur would sometimes have cpu hiccups.

gaaclarke · 2024-10-04T16:52:23Z

Funny thing though is there is a bug in skia like the one I had to find for rendering small sigmas.

Skia issue filed: https://g-issues.skia.org/issues/371571787

gaaclarke

lgtm

gaaclarke · 2024-10-04T18:40:15Z

@jonahwilliams do you want to give this another look? I've contributed to it quite a bit so might be good to have @flar look over my additions or another person. I tried to avoid touching jims code as much as possible. I made the following changes:

Fixed ios compilation
Fixed bug with small sigmas
Added a test that show we don't use this for ovals
Added a test to show the smalls sigmas work now

gaaclarke · 2024-10-04T18:41:07Z

I'm not sure what is going on with the golden images. They aren't showing up yet but I think they should have been processed. I looked at previous versions of this PR and they all looked good to me except when the corner radii are not equal, of course.

jonahwilliams · 2024-10-04T18:41:25Z

Taking a look

jonahwilliams · 2024-10-04T18:49:38Z

They bot won't ding until Mac mac_unopt finished. Otherwise maybe they are identical?

flutter-dashboard · 2024-10-04T19:29:05Z

Golden file changes have been found for this pull request. Click here to view and triage (e.g. because this is an intentional change).

If you are still iterating on this change and are not ready to resolve the images on the Flutter Gold dashboard, consider marking this PR as a draft pull request above. You will still be able to view image results on the dashboard, commenting will be silenced, and the check will not try to resolve itself until marked ready for review.

Changes reported for pull request #55604 at sha e511f69

jonahwilliams · 2024-10-04T19:33:10Z

A suggestion based on @gaaclarke 's work to vectorize the previous blur:

diff --git a/impeller/entity/shaders/rrect_blur.frag b/impeller/entity/shaders/rrect_blur.frag
index d0bba43561..c5368daf6a 100644
--- a/impeller/entity/shaders/rrect_blur.frag
+++ b/impeller/entity/shaders/rrect_blur.frag
@@ -12,7 +12,7 @@ precision highp float;
 #include <impeller/types.glsl>
 
 uniform FragInfo {
-  f16vec4 color;
+  vec4 color;
   vec2 center;
   vec2 adjust;
   float minEdge;
@@ -26,7 +26,7 @@ frag_info;
 
 in vec2 v_position;
 
-out f16vec4 frag_color;
+out vec4 frag_color;
 
 const float kTwoOverSqrtPi = 2.0 / sqrt(3.1415926);
 
@@ -34,19 +34,17 @@ float maxXY(vec2 v) {
   return max(v.x, v.y);
 }
 
-// use crate::math::compute_erf7;
-float computeErf7(float x) {
-  x *= kTwoOverSqrtPi;
-  float xx = x * x;
-  x = x + (0.24295 + (0.03395 + 0.0104 * xx) * xx) * (x * xx);
+vec2 computeErf7(vec2 x) {
+  x *= vec2(kTwoOverSqrtPi);
+  vec2 xx = x * x;
+  x = x + (vec2(0.24295) + (vec2(0.03395) + vec2(0.0104) * xx) * xx) * (x * xx);
   return x / sqrt(1.0 + x * x);
 }
 
 // The length formula, but with an exponent other than 2
 float powerDistance(vec2 p) {
-  float xp = pow(p.x, frag_info.exponent);
-  float yp = pow(p.y, frag_info.exponent);
-  return pow(xp + yp, frag_info.exponentInv);
+  vec2 pp = pow(p, vec2(frag_info.exponent));
+  return pow(pp.x + pp.y, frag_info.exponentInv);
 }
 
 void main() {
@@ -55,9 +53,10 @@ void main() {
   float dPos = powerDistance(max(adjusted, 0.0));
   float dNeg = min(maxXY(adjusted), 0.0);
   float d = dPos + dNeg - frag_info.r1;
+
+  vec2 erfs = computeErf7(vec2(frag_info.sInv * (frag_info.minEdge + d), frag_info.sInv * d));
   float z =
-      frag_info.scale * (computeErf7(frag_info.sInv * (frag_info.minEdge + d)) -
-                         computeErf7(frag_info.sInv * d));
+      frag_info.scale * (erfs.x - erfs.y);
 
-  frag_color = frag_info.color * float16_t(z);
+  frag_color = frag_info.color * z;
 }

jonahwilliams · 2024-10-04T19:34:32Z

impeller/display_list/canvas.cc

@@ -498,6 +498,11 @@ bool Canvas::AttemptDrawBlurredRRect(const Rect& rect,
    return false;
  }

+  // The current rrect blur math doesn't work on ovals.
+  if (fabsf(corner_radii.width - corner_radii.height) > kEhCloseEnough) {


Instead of adding this case you could also adjust the logic in IsNearlySimpleRRect to force all corner radii to be equal.

But that would also eliminate fast tessellation of simple rrects. The only deficiency here is that we can no longer fast_blur them so it's a test specific to this method.

agh, yes don't do that then! :)

jonahwilliams

LGTM

gaaclarke · 2024-10-04T20:18:25Z

A suggestion based on @gaaclarke 's work to vectorize the previous blur:

Jim and I were looking at this a bit yesterday and I poked around a bit this morning. We should do this in a follow up PR to establish this as the baseline.

flar · 2024-10-04T20:29:52Z

A suggestion based on @gaaclarke 's work to vectorize the previous blur:

Jim and I were looking at this a bit yesterday and I poked around a bit this morning. We should do this in a follow up PR to establish this as the baseline.

Another optimization is that we don't need to subtract the center inside the shader, that should be done by changing the initial mapping of the coordinates, then the shader just needs to do abs(position) - adjust.

Actually, if we were willing to emit 4 quads then we could do that abs(position) - adjust on the CPU by setting up the 4 quads with their absolute (adjusted) values.

jonahwilliams · 2024-10-04T20:32:16Z

I would do the subtraction in the vertex shader, that would only execute ~4 times or so. But the subtraction is likely not going to add much overhead

flar · 2024-10-04T20:39:03Z

I would do the subtraction in the vertex shader, that would only execute ~4 times or so. But the subtraction is likely not going to add much overhead

That's not needed, you just have to change 4 values in the C++ code and all of the coordinates will be correct without any more work in the vertex shader. Right now it has to do math to make everything relative to the rect origin, just swap in rect center instead and it's good to go.

Could the vertex shader convert the 1 quad into 4 to do the quadrant slicing? Actually if it could slice it (1 -> 9) then we could get rid of the abs, the -adjust and the subsequent elimination of the negative values in the fragment shader.

jonahwilliams · 2024-10-04T20:42:53Z

You could slice up the geometry however you'd like

jonahwilliams · 2024-10-04T20:43:34Z

If you want data to vary per vertex or group of vertexes you'd usually do that by defining a new varying instead of using a uniform.

flutter/engine#55604)

gaaclarke · 2024-10-04T22:29:55Z

So I tried to vectorize this and tried some other tricks. Vectorization didn't pay off. The patch above actually makes things slower, but for a large part of the output space powerDistance is just a slow version of length when the exponent is 2. This happens for rendering circles or any sufficiently blurred rrect. This is 15% faster than what we landed with an average gpu time of 3.82ms for the benchmark.

I tried to branch (if exponent == 2.0)) in the fragment shader but that wasn't any faster. We would have to probably maintain a separate fragment shader for this special case.

For reference, our exponents appear to be in the range [2, 3.5].

--- a/impeller/entity/shaders/rrect_blur.frag
+++ b/impeller/entity/shaders/rrect_blur.frag
@@ -44,9 +44,13 @@ float computeErf7(float x) {
 
 // The length formula, but with an exponent other than 2
 float powerDistance(vec2 p) {
-  float xp = pow(p.x, frag_info.exponent);
-  float yp = pow(p.y, frag_info.exponent);
-  return pow(xp + yp, frag_info.exponentInv);
+    return length(p);
 }

gaaclarke · 2024-10-04T22:34:53Z

You could also use specialization constants to avoid having a separate shader but those only exist in vulkan I think.

jonahwilliams · 2024-10-04T22:38:12Z

spec constants are supported for all backends. I would not pursue a specialization constant for this.

gaaclarke · 2024-10-04T22:40:34Z

spec constants are supported for all backends. I would not pursue a specialization constant for this.

What's your take in general on trying to get that 15% boost on circles and really blurry rounded rects? I know adding shaders has a maintenance and startup time cost.

jonahwilliams · 2024-10-04T22:41:25Z

Doesn't seem worthwhile

…156252) flutter/engine@eece6c3...bc5f9fc 2024-10-04 flar@google.com [Impeller] Use a squircle-sdf-based algorithm for fast blurs (flutter/engine#55604) 2024-10-04 skia-flutter-autoroll@skia.org Roll Skia from 59f512b47cc0 to 4aff9603622d (1 revision) (flutter/engine#55663) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll Please CC matanl@google.com,zra@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

gaaclarke · 2024-10-04T22:59:55Z

For future reference I measured with the fragment shader just setting a solid color and the GPU time was 1.6ms, so that's the floor. Considering that; the length variant is 24% faster (1-(3.82-1.6)/(4.54-1.6)).

Doesn't seem worthwhile

I'm fine with that estimation, we can always revisit if we need to.

flar · 2024-10-04T23:16:24Z

So I tried to vectorize this and tried some other tricks. Vectorization didn't pay off. The patch above actually makes things slower, but for a large part of the output space powerDistance is just a slow version of length when the exponent is 2. This happens for rendering circles or any sufficiently blurred rrect. This is 15% faster than what we landed with an average gpu time of 3.82ms for the benchmark.

That's the definition of a squircle and why this algorithm works. Yes, we can replace those powers by "2" and just use length, but then they aren't plotting a squircle, they are plotting a circle - and a circle is not an approximation of a blurred circle/rrect.

Ah, I see, you are suggesting testing the constant to see if it happens to be "2", how often is that the case?

gaaclarke · 2024-10-04T23:29:10Z

Ah, I see, you are suggesting testing the constant to see if it happens to be "2", how often is that the case?

The exponent is 2 when:

if corner_radius >= 50
if corner_radius = 25 && blur_radius >= 26
if corner_radius = 1 && blur_radius >= 30.9

So, about half of the output space.

gaaclarke · 2024-10-04T23:42:07Z

We got the official perf results: https://flutter-flutter-perf.skia.org/e/?queries=test%3Drrect_blur_perf_ios__timeline_summary&selected=commit%3D42818%26name%3D%252Carch%253Dintel%252Cbranch%253Dmaster%252Cconfig%253Ddefault%252Cdevice_type%253DiPhone_11%252Cdevice_version%253Dnone%252Chost_type%253Dmac%252Csub_result%253Daverage_gpu_usage%252Ctest%253Drrect_blur_perf_ios__timeline_summary%252C

44% faster than the old rrect_blur, 37% faster than the vectorized rrect_blur.

jonahwilliams · 2024-10-05T00:25:31Z

This is a phenomenal improvement - Great work @flar + @gaaclarke !

…/engine#55604) Use a Squircle Signed Distance Field based algorithm to do a very fast approximation of rrect blurs. This PR is to provide reference within the team to discuss the future of the algorithm compared to the Gaussian approximation functions that are currently in use. It isn't a complete solution, but can be completed easily with a little more work. Notably, it doesn't handle elliptical round rects, only circular corners. Could stand to include an attribution to the source (https://raphlinus.github.io/graphics/2020/04/21/blurred-rounded-rects.html)

[Impeller] Use a squircle-sdf-based algorithm for fast blurs

6c0a0ea

flar requested a review from gaaclarke October 3, 2024 03:25

github-actions bot added the e: impeller label Oct 3, 2024

fixed ios builds

2f0fd1d

added playground

b0689e8

fixed small sigma render problem by expanding the existing clamp

356a79f

added fallback to slow path for ovals

876bc4f

gaaclarke added 4 commits October 4, 2024 10:07

added test that makes sure to hit the new rrect_blur for small sigmas

fcb764a

added comment to source

83fe465

removed the autos

a6399e8

updated malioc

85fb8da

gaaclarke approved these changes Oct 4, 2024

View reviewed changes

gaaclarke requested a review from jonahwilliams October 4, 2024 18:38

flutter-dashboard bot added the will affect goldens label Oct 4, 2024

jonahwilliams reviewed Oct 4, 2024

View reviewed changes

jonahwilliams approved these changes Oct 4, 2024

View reviewed changes

gaaclarke added the autosubmit Merge PR when tree becomes green via auto submit App label Oct 4, 2024

auto-submit bot merged commit bc5f9fc into flutter:main Oct 4, 2024
33 checks passed

engine-flutter-autoroll mentioned this pull request Oct 4, 2024

Roll Flutter Engine from eece6c3ec63a to bc5f9fc80b06 (2 revisions) flutter/flutter#156252

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Oct 4, 2024

bc5f9fc80 [Impeller] Use a squircle-sdf-based algorithm for fast blurs (

43385af

flutter/engine#55604)

gaaclarke mentioned this pull request Oct 4, 2024

[Impeller] Impeller RRect blur is slower than Skia's cached blur shadows. flutter/flutter#148496

Closed

1 task

gaaclarke mentioned this pull request Oct 9, 2024

[Impeller] rrect blur has weird artifacts sometimes flutter/flutter#152887

Closed

flar mentioned this pull request Jan 28, 2025

[iOS] [Impeller] Flutter 3.27 BoxShadow.spreadRadius draws black box artefact around widget on physical iOS device, not on simulator flutter/flutter#162128

Closed

[Impeller] Use a squircle-sdf-based algorithm for fast blurs #55604

[Impeller] Use a squircle-sdf-based algorithm for fast blurs #55604

Uh oh!

Conversation

flar commented Oct 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

flutter-dashboard bot commented Oct 3, 2024

Uh oh!

gaaclarke commented Oct 3, 2024

Uh oh!

gaaclarke commented Oct 3, 2024

Uh oh!

gaaclarke commented Oct 3, 2024

Uh oh!

gaaclarke commented Oct 3, 2024

Uh oh!

gaaclarke commented Oct 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

expected output

seen output

Uh oh!

gaaclarke commented Oct 3, 2024

Uh oh!

gaaclarke commented Oct 3, 2024

Uh oh!

gaaclarke commented Oct 3, 2024

Uh oh!

gaaclarke commented Oct 4, 2024

before

after

skia

Uh oh!

gaaclarke commented Oct 4, 2024

Uh oh!

gaaclarke commented Oct 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gaaclarke commented Oct 4, 2024

Uh oh!

gaaclarke left a comment

Choose a reason for hiding this comment

Uh oh!

gaaclarke commented Oct 4, 2024

Uh oh!

gaaclarke commented Oct 4, 2024

Uh oh!

jonahwilliams commented Oct 4, 2024

Uh oh!

jonahwilliams commented Oct 4, 2024

Uh oh!

flutter-dashboard bot commented Oct 4, 2024

Uh oh!

jonahwilliams commented Oct 4, 2024

Uh oh!

jonahwilliams Oct 4, 2024

Choose a reason for hiding this comment

Uh oh!

flar Oct 4, 2024

Choose a reason for hiding this comment

Uh oh!

jonahwilliams Oct 4, 2024

Choose a reason for hiding this comment

Uh oh!

jonahwilliams left a comment

Choose a reason for hiding this comment

Uh oh!

gaaclarke commented Oct 4, 2024

Uh oh!

Uh oh!

flar commented Oct 4, 2024

Uh oh!

jonahwilliams commented Oct 4, 2024

Uh oh!

flar commented Oct 4, 2024

Uh oh!

jonahwilliams commented Oct 4, 2024

Uh oh!

jonahwilliams commented Oct 4, 2024

Uh oh!

flar commented Oct 3, 2024 •

edited

Loading

gaaclarke commented Oct 3, 2024 •

edited

Loading

gaaclarke commented Oct 4, 2024 •

edited

Loading

flar commented Oct 4, 2024 •

edited

Loading

gaaclarke commented Oct 4, 2024 •

edited

Loading