Shader-based BLUR Filter Discrepancy in p5.js

[SableRaf]

Increasing Access

Ensuring a consistent visual output across p5 versions reduces confusion and lessens cognitive load for both learners and long time users.

Most appropriate sub-area of p5.js?

• Accessibility
• Color
• Core/Environment/Rendering
• Data
• DOM
• Events
• Image
• IO
• Math
• Typography
• Utilities
• WebGL
• Build Process
• Unit Testing
• Internalization
• Friendly Errors
• Other (specify if possible)

Feature enhancement details

Note: this is a follow up to the conversation on #6324

First of all, many thanks to @wong-justin, @aferriss, and @aceslowman for the outstanding work on integrating shader-based filters in p5.js.

However, I've noticed a difference in the output of the filter(BLUR). For clarity, I've put together a visual comparison using a small demo sketch available here. Additionally, here's the image derived from the demo:

Generated on Chrome Version 116.0.5845.140 (Official Build) (arm64), MacOS Ventura 13.4 (22F66)

The newly added shader for BLUR uses a box blur, as opposed to the Gaussian blur in CPU mode. Might be worth checking if we can use a Gaussian kernel in the shader and what that means for performance. It'd be great to have outputs that are consistent, especially for older projects that were built around the original blur and whose outputs will look different with the new shader-based method.

Looking forward to the community's insights on this.

[wong-justin]

I think blur is the hardest filter to match exactly with CPU mode.

The main difficulties trying to recreate the Gaussian blur were:

• making a kernel size that could change on any frame according to the user's parameter, which could mean hard coding many different kernels in one shader, or compiling on the fly in the middle of runtime

• alternatively using a small kernel and repeating passes for more blur, but the extra passes were slowing things down with all the back-and-forth between layers

Happy to hear other thoughts and approaches though. It would be nice to have a better blur.

[davepagurek]

Instead of having a hardcoded kernel, we could generate the kernel value dynamically per sample based on the distance from the center. I'm not sure how much of a performance impact this would have compared to using hardcoded values, but if it's not significant, it would avoid the complexity of managing different shaders.

To loop over a variable number of pixels, to circumvent the issue where loops have to have constant bounds, I generally see people loop up to a max value (max radius, in this case) and then break if the loop index equals the real loop limit.

[SableRaf]

I asked Patricio Gonzales-Vivo (creator of Lygia and co-creator of the Book of Shaders). He shared the following insights:

If matching what’s on the cpu is the goal you could upload the weight you use them to procedurally write the kernel convolution shader. Also for reference in lygia I calculate them on the fly https://lygia.xyz/filter/gaussianBlur/2D Is not incredibly performant but is not the worst on the world. Is the goal efficiency or accuracy?

I'd argue that accuracy should be essential since we're replacing the CPU blur with the GPU blur. I believe minimizing discrepancies would be beneficial. As for performance, we could test it and see how costly this type of blur is. I'm guessing it should still be significantly faster than CPU blur but some benchmarking would help 🤓

An example of all 2D shaders in Lygia, including 2D gausian blur, using p5.js can be found here: https://editor.p5js.org/patriciogonzalezvivo/sketches/XCkTzoyB3

[wong-justin]

Ah ok, thanks for the resources and ideas. I can take the next week or two to dig in and learn and see what I can come up with.

[SableRaf]

Just one more resource that may be helpful:

https://github.com/SableRaf/Filters4Processing/blob/master/sketches/GaussianBlur/data/gaussianBlur.glsl

Taken from: http://callumhay.blogspot.com/2010/09/gaussian-blur-shader-glsl.html

[wong-justin]

Here's an initial draft: https://editor.p5js.org/jwong/sketches/Cn2z2G-to
The weight function needs some adjusting, and I haven't thought about matching CPU output yet. Supposedly with the right math, I think it could be tuned pretty close to the original.

But this is a single shader, single pass gaussian blur using the break technique (I think I've wrapped my head around it now thanks to everybody's help)

Note that performance drops kinda quickly. Some rough manual benchmarks from my machine:

steps / kernel size / parameter -->	10	20	30	40
current shader BLUR, fps	60	60	57	50
new linked BLUR, fps	60	30	10	7

[aferriss]

I think the current implementation could be modified to continue using the two pass solution, which should keep things much more performant. Here's another implementation of the gaussian kernel setup for a two pass renderer

https://editor.p5js.org/aferriss/sketches/IS4P5RdAo

[SableRaf]

Thanks @wong-justin and @aferriss! How do these two approaches compare with the CPU blur in terms of quality?

[aferriss]

Here's a few comparisons I put together, with the GPU blur on top and CPU on the bottom of each image. None of them match perfectly, and to be honest I think we will have a pretty difficult time matching the cpu blur exactly. I think we may need to make a tradeoff here between performance and visual appeal. That being said we can keep tinkering away at this and hopefully get as close as possible. I think of the options presented here the one I shared yesterday seems the closest, but would love to hear other opinions!

[SableRaf]

@aferriss I agree that your version looks the best of the ones there.

Another approach suggested here is to use a single precalculated kernel and apply the blur multiple times with two ping-pong framebuffers.

I've made a quick attempt based on your sketch, setting the samples relatively low, but calling filter(s) multiple times per frame. The results seem really encouraging in terms of both performance and visual quality.

Edit: I also tried a version of the hardcoded kernel from the article: https://editor.p5js.org/SableRaf/sketches/NTy_zPyPX

[SableRaf]

Another useful reference. This article does a systematic exploration of GPU blur, with regard to performance and quality across platforms:
https://venturebeat.com/pc-gaming/an-investigation-of-fast-real-time-gpu-based-image-blur-algorithms/

[SableRaf]

I've done further exploration using a shader to visualize the lightness as a graph, inspired by the figures in the article from VentureBeat. You can find the test sketch here: https://editor.p5js.org/SableRaf/sketches/6it1LpFq8

We can see that Adam's blur is pretty close but it has a bias towards the bottom right direction that becomes more visible at higher samples.

The multipass blur is also quite close but it explodes at extreme number of passes, I suppose because of edge effects. (edit: I'm not sure about that image, looks like I may have made a mistake while generating it, I'll have to get back to it)

This is not a super rigorous benchmark but I hope it will be helpful.

By the way, I'm testing in Chrome Version 116.0.5845.187 on macOS Ventura 13.4.

CPU Blur filter(BLUR,20,false) (ground truth)

Current GPU Blur filter(BLUR,20)

Multipass blur (200 passes)

Adam's blur (60 samples)

Adam's blur (80 samples)

Multipass blur (400 passes)

[davepagurek]

I think the offset in Adam's might be from this line:

float x2 =  (i - samples * 0.5);

If we're using e.g. samples=3, then we get:

i	x2
0	-1.5
1	-0.5
2	1.5

If we instead use x2 = i - (samples-1.) * 0.5, we get:

i	x2
0	-1
1	0
2	1

This only works when samples is odd, but if we define samples = blurRadius * 2 + 1 (so a radius of 1 means it averages the pixel to the left and the pixel to the right with the current pixel) then this would always be true.